In a recent conversation about “what are you up to these days?”, I mentioned that I’m currently mainly focused on AI Engineering. The response I got was: “But isn’t AI just typing in a question?”
I thought to myself, “How am I going to explain this?”, and realised I actually wanted to have it clearly defined for myself as well. On top of that, I’d love to be able to send that person a link straight away with the message, “Just check this out.”
Well, here you go. Check this out:
An AI Engineer is a software engineer who integrates existing AI models with data sources, external tools and applications to build reliable, useful and scalable AI-powered applications.
Something like this? :)
The term AI Engineer is relatively new. But what exactly does the profession involve? What skills do you need to become an AI Engineer? And what can an AI Engineer actually build?
A further explanation based on current insights:
From Programmer to AI Engineer
In the past, becoming a programmer required quite a journey. You completed an education, did an internship, worked on assignments and gained experience before eventually being able to start working as a programmer.
That landscape is changing rapidly. With the arrival of AI coding tools, writing code has become much more accessible. AI can now generate large parts of the programming work for you.
But that doesn't mean the difficult part of software development has disappeared. It has shifted.
The biggest challenge lies in making the right decisions:
- Architecture: How do you structure an application, and which technical architecture is best suited to the solution?
- Problem definition: What exactly do you need to build to solve a problem?
- Trade-offs: Why is one technical approach better than another?
These are skills you don't simply acquire through an education or training course. They mainly develop through experience, experimentation and actually building things.
This is precisely where the AI Engineer comes into play.
Machine Learning (ML) Researcher vs. AI Engineer
To understand what an AI Engineer does, it is also useful to understand the distinction between a Machine Learning Researcher and an AI Engineer.
A Machine Learning Researcher focuses on developing AI models themselves. This includes:
- Building AI models.
- Training AI models.
- Designing new LLM architectures.
- Publishing scientific papers.
- Researching the underlying technology.
This is generally a specialised field that requires a strong academic background, often in mathematics.
An AI Engineer, on the other hand, builds systems using existing AI models. Think of frontier models or open-source models developed and trained by Machine Learning Researchers.
The work includes, among other things:
- Integrating AI models into existing or new platforms.
- Connecting models to databases and other data sources.
- Connecting AI models to tools to retrieve external information and perform actions.
- Setting up MCP servers and connecting AI Agents.
- Implementing guardrails and memory loops for agents.
- Making AI solutions available within an application or organisation.
- Lots more. See, for example, Bringing AI into production.
You can compare it to the automotive industry: the Machine Learning Researcher builds the engine, while the AI Engineer builds the car.
There are now countless powerful AI models available. The challenge increasingly lies in building the cars: developing genuinely useful solutions that organisations can actually work with.
There is currently a great demand for AI Engineers in this area.
The Skill Set Required to Become an AI Engineer
1. Fundamental Technical Skills
A common mistake is to immediately start building an AI integration and putting it into use without properly understanding the underlying technology and data.
The foundation is then skipped and has to be put in place afterwards.
This foundation consists of technical skills that are not specifically related to AI, but are essential for building good AI applications.
Python
You don't have to be a Python wizard, but you do need to understand what is happening in the code that an AI coding agent writes for you.
Many Machine Learning libraries and packages use Python, including PyTorch and TensorFlow. Understanding the language is therefore important.
Git
Git is essential for version control. You need to be able to work with repositories, track changes and understand how code is managed.
CLI tools and Linux
You need to be comfortable navigating the Command Line Interface (CLI) and understand the basics of Linux.
Git, CLI tools and Linux are also important when deploying your AI application.
APIs
APIs are perhaps one of the most important components of AI Engineering. They allow you to connect different software, models and data sources.
For example, when building AI integrations, you make API calls to:
- AI models.
- Vector databases.
- Relational databases.
- External tools and services.
- Search engines, e.g. Solr.
You also need to know how to process responses, handle rate limits and take token budgets into account. Commercial AI models charge based on the number of tokens you consume. Think of OpenAI's or Claude's frontier models.
Ultimately, a good AI solution consists of a correctly structured application in which different APIs communicate with each other.
From your own application to the AI model, and then to various tools, databases and external services.
A huge amount of data is constantly flowing in all directions. It is important to understand how this communication works.
2. AI Skills Required for an AI Engineer
In addition to the technical foundation, you naturally also need specific AI knowledge. Some important components include embeddings, vector search, RAG and AI agents.
Embeddings and Vector Search
To enable AI models to work with large amounts of data, you need to understand what embeddings are and how vector search works.
For example, embeddings convert text from PDF files into numerical vectors. This allows a system to search based on meaning (semantic similarity) rather than relying solely on exact keywords.
You can then perform a search in the vector database to retrieve the most relevant and recent information and feed it into the context window of the relevant AI model, together with the human user's original question.
The AI model / LLM can now formulate an accurate answer because the most recent and relevant information has been provided.
Through this process, an end user can, for example, ask a question about a document without using exactly the same words as the original text and receive a useful answer based on semantic similarity.
RAG: Retrieval-Augmented Generation
Another important component is RAG. This allows you to give an AI model access to relevant and up-to-date information so that it can provide answers based on the correct data sources.
AI models are trained on data up to a certain point in time. They are not continuously updated with all new information. RAG helps bridge that gap.
The RAG process in broad terms:
- PDF: You start with documents and other data sources.
- Chunks: The documents are divided into smaller text fragments.
- Embeddings: Embeddings are created from these fragments.
- Vector database: The embeddings are stored in a vector database.
- Search query: An end user asks a question. The system searches the vector database for the most relevant information.
- Context: The retrieved information is added to the question and sent to the AI model.
- Answer: The AI model uses this context to generate an answer in understandable language.
This gives the model access to information that was not necessarily part of its original training data.
For many organisations looking to get started with AI, RAG or a variation of it is an important part of the solution.
AI Agents and Tools
AI agents are currently very popular. Whereas a traditional AI model primarily generates answers, agents can also actually perform actions. Often actions similar to those performed by humans.
This is achieved by connecting AI models to tools. For example:
- Retrieving information from a large, complex database.
- Performing an action in another system.
- Retrieving external information.
- Or analysing a spreadsheet.
This shifts AI from simply providing answers to actually carrying out work.
The Difference Between an AI Workflow and an AI Agent
When building AI solutions, it is also important to understand the difference between workflows and agents.
With an AI workflow, steps are predefined and executed in a specific order. With an AI agent, the agent loop plays an important role: based on the available information, the model determines which next step is required, which tool it should use and whether the result is sufficient.
This loop is crucial. If you can make it function reliably and at scale, you can create a powerful AI system.
3. Deployment Skills Required for an AI Engineer
Building an AI solution is one thing. Ensuring that it works reliably in a production environment is another.
That is why deployment skills are also essential.
Containers
You need to ensure that an AI application can be reliably transferred from your local development environment to a production environment. You also need to make sure that all the developers involved can work with it without constantly having to install and configure their local environments. You want them to be able to get started immediately with development, regardless of their OS.
Monitoring
AI applications involve enormous amounts of data flowing in all directions. You therefore want to have insight into what is happening within your system.
Monitoring helps you, among other things, to:
- Understand why an AI agent made a particular decision.
- Gain insight into how your AI system operates.
- Track the number of tokens used and monitor the financial impact of AI usage.
- Ensure transparency and trust in the system.
Local Servers and Cloud
Finally, you need to understand how to make AI solutions available to human users.
This can be done, for example, by using in-house or on-premise servers, where systems operate independently of the internet. Alternatively, applications can be hosted in the cloud so that they are available worldwide.
What Are AI Engineers Currently Being Used For?
These skills come together in various solutions that organisations are increasingly having developed.
AI Knowledge Systems
Systems in which employees can ask questions and receive answers based on all the knowledge and data within the organisation.
AI Agents and Tools
Agents that can, for example, query a database, visualise data or perform tasks that would normally require the knowledge of a specialist.
Developing and Deploying Software / Applications
Using AI to accelerate software development. Developing and deploying applications that previously might have taken weeks can sometimes be achieved within just a few hours with the help of AI.
And More
Also see AI Engineering > What AI can mean for your digital platform.
Finally
AI Engineering is therefore about much more than simply entering a question into ChatGPT.
It is about combining existing AI models with software, data sources, tools and infrastructure. It requires technical knowledge, but above all, the ability to make the right decisions: what do you build, how do you build it, how do you ensure that it works reliably and continues to work? How do you prevent costs from getting out of hand and ensure that end users remain satisfied? And lots more.
AI is making writing code increasingly accessible. But building good, useful and scalable systems requires insight, experience and technical skills.
And that is precisely what AI Engineering is all about.
Related blog
Bringing AI into production: the real challenge begins after the demo
