With AI, great product engineering is relevant more than ever.

Anton Keks and I sit together again to explore the real question: what does it mean to be an engineer now, in today's AI world?

In this round 2 conversation, we talk about the shift from coding to thinking — from generating output to making decisions. Because AI doesn’t remove the need for engineering fundamentals; it amplifies the cost of not having them.

We cover what AI engineering actually looks like in practice: how to use these tools without losing control, how to avoid shallow understanding, and why system design, feedback loops, and clean abstractions matter more than ever.

We delved deep into discussing pair and pair-programming practices in the age of AI, and how engineers can collaborate with each other to avoid knowledge silos.

We also get into the operational side — CI/CD, security, engineering workflows, and building systems that scale and survive.

If you’re leading teams or writing production code in 2026, this is the podcast you might be interested to listen to.

00:00 Intro & What This “Round 2” Is About
00:10:55 Blind Execution vs Real Understanding in Engineering
00:21:50 Why Core Engineering Skills Never Go Away
00:32:45 Avoiding Knowledge Silos in Teams
00:43:40 Team Culture, Demos & Feedback Loops
00:54:35 Reducing Boilerplate & Writing Cleaner Code
01:05:30 Productivity Tradeoffs & Time Optimization
01:16:25 Using AI Effectively as an Engineer
01:27:20 CI/CD, Automation & Developer Workflows
01:38:15 Security, Vulnerabilities & Real Risks
01:49:10 Building Systems That Last for Years
02:00:05 Scaling Systems & Working with Subdomains

Thanks for listening. Feel free to subscribe on YouTube or on MapForEngineers.com substack. See you next time.

Subscribe now

Here are the slides that I used for my talk on AI Agents at PyCon Estonia.

Disclaimer: I am not affiliated with any company that I mention below (except Pactum, where I work as head of product engineering). All opinions below are my own.

—

I haven’t been to any engineering-focused conference for a while, so I decided to go to one. So Carl and I were flying from Tallinn to visit the conference in Paris, with the goal to, hopefully, bring back learnings back to Pactum (where both of us work) for our engineering org. I was afraid there would be a lot of sales people who will only sell me things, and not much “meat”. I am happy it turned out the other way around - a lot of cool engineers & AI enthusiasts visited the conference, from Nvidia, Shopify, Sentry, BMW, Mistral, Spotify etc. All right, let’s dive into the learnings!

All themes revolved around building and deploying AI agents.

Building and optimising data layer for AI agents

CEO of Neo4J, Emil Eifrem, talked about how they think about building a data layer optimised for agents. His talk was actually pretty technical and good. Emil’s main point was that data layer for AI agents would have to have these 4 properties:

1. Handle structured, unstructured, and semi-structured data.

2. Consistently and reliably extract entities from unstructured data.

3. Link entities across memory and application data.

4. Disambiguate first party and derived data.

Emil was honest that in Neo4J they are pretty much still building out those properties optimised for AI agents usage.

My takeaway. It’s an interesting idea to use native graph database as a knowledge graph, to do advanced techniques such as GraphRAG on top of it etc. Right now my knowledge on Neo4J and graph databases is limited, but my approach, as an engineer, would be the following regarding the data layer for AI agents:

1. First, start simple. Understand how you will do evals of your agentic features. Now you have a feedback loop. Here goes your test-first approach.

2. And if you use Postgres, for example, stick to it, for now. First, throw all you need into a context window. If that doesn’t work, try giving your agent useful tools (e.g. exposed over MCP) to query your data. That might be a powerful enough and simple technique for your agent to read your data.

3. If giving your agent tools that manipulate your data layer is not good enough, as reflected by your evals test pipeline, try RAG with pgvector. Optimise RAG parameters (chunking, embedding model etc). Your evals will tell you if you move in the right direction.

4. That didn’t work either? Step back. Think about your agentic workflows. Can you split your big use-case into smaller steps? Agan, think about agentic workflow primitives.

5. Did I mention that by now you probably need to ship your feature? If your test evals are good enough, ship, and learn from your users and results. Definitely have monitoring and tracing and check evaluation results.

6. If you have all those basic steps in place with solid feedback loops and you want to optimise even further, sure, go ahead and explore graph databases and cool things like GraphRAG. But don’t over-engineer at first. Move with small, safe steps, optimising only when strong feedback loops are in place.

Did I mention AI evals multiple times? Let’s explore them in more detail!

AI Evals: what’s now and what’s coming

Code is deterministic. AI agents using LLMs are usually non-deterministic. Mastra defines evals in the following way (source):

Evals are automated tests that evaluate Agents outputs using model-graded, rule-based, and statistical methods.

I chatted to CTO of Orq.ai at the conference, and engineers from Arize.com. Both companies are focusing on building out infra and tools for evals as part of their AI platform offering. Orq, for example, builds sophisticated evals guided by agent that is able to evaluate multiple steps of your agent run. Arize also has a lot of cool tooling, for different use-cases of evals. Both Arize and Orq are actively iterating on their evals product, learning based on how their customers use evals. There is no established single set of “best practices” yet, only active exploration.

@swyx, who gave a talk at the conference, said that he knows that Claude Code engineers still say that “Claude Code evals are still a lot about the vibes” [everybody laughed at the room]. It was semi-joke, but with a point. Swyx, I think, wanted to underline the point that the industry as a whole is still learning about how to do AI evals. It’s art & science and the wild west still.

My takeaway. Everyone is learning and exploring how to do AI evals, as it’s totally a different beast from a traditional testing approach. One thing is clear though - it’s crucial to test your agents and gain feedback loops, both in your CI/CD pipeline and in production through observability. AI evals is a complex topic and is 100% worth exploring and learning about if you are serious about deploying AI agents to production. Just to give you a sense of techniques, below is a visualization from open-source engineering platform, Langfuse, on online & offline flows of AI evals and how they interact together:

System prompt engineering via meta prompting and explanations from AI evals

There was a really cool talk by co-founder of Arize, Aparna Dhinakaran. It opened my mind how you can do system prompt engineering at scale in a data-driven way. Here is the flow:

1. Let’s say you have v1 of your system prompt for your AI agent. If you deploy a customer support AI agent, maybe your system prompt says something along the lines of “You are a helpful customer support agent. Try to help customers to resolve the issue”. I am oversimplifying with this description of course. Usually your system prompt would include a lot of instructions and examples as well. The whole point is - it’s your first version, and over time you would like to improve your system prompt. How do you do it?

2. You deploy your AI evals, for both offline (CI/CD) and online (production observability). Important bit now - your AI evals include not only some score (be it label or a normalised score from 0 to 1), but also an explanation.

3. Now here is a cool bit. After your AI agent got some traffic, you will have a lot of those AI evals with scores and explanations. For example, if one agent output was not so good, your AI eval might give it a score with label “bad” or “0” with an explanation: “Agent output did not clarify the problem of the customer”.

4. You then write a meta prompt with the goal to improve the system prompt. You feed all corpus of generated explanations from AI evals into your meta prompt, and your meta prompt generates a better system prompt.

   1. For example, your system prompt might look like this (oversimplifying of course, just to show the point):
      ”””
      Your goal is to improve a system prompt. I will give you my current system prompt and all explanations from AI evals. Based on this input, please generate new improved version of system prompt, taking into account feedback from explanations from AI evals.
      Current system prompt: <system prompt>
      Explanations from AI evals results: <array of explanations>
      ”””

Voila! You have your second iteration of a system prompt. Now you can evaluate your system prompt with your offline AI evals and if it improves performance, you deploy it to production. Once you gather enough production data of explanations from AI evals, you can re-run meta prompt again to improve a system prompt further.

This is data-driven way to improve your system prompt!

BMW explores GUI-based Agents

BMW gave a talk about how they explore GUI-based agents to manipulate their Android UI. It might be useful for some use-cases, where usage via strict API is not practical for one reason or another. GUI agents or browser-based agents might be a hybrid way to interact with some external system that you don’t have full control over or the one that changes rapidly. Just keep it in mind and consider using this tool if it helps you solve some integration problem that you have.

LlamaIndex gave a cool talk on cloning NotebookLM. Structured data extraction

My key takeaway is the following: the industry has solved the use-case of structured data extraction pretty well. To clarify, I mean the mapping of unstructured data to a structured one.

As stated on IBM docs on one of prominent use-cases for LlamaIndex:

Structured data extraction: LLMs process natural language and extract semantically significant details such as names, dates, addresses and figures and present them in a consistent, structured format regardless of the original source. Once the data is structured, it can be sent to a database or further analyzed with tools such as LlamaParse.

This use-case is very important for the whole industry. For example, in Pactum, we had to solve the same problem of mapping of unstructured data to a structured one. We used Gemini model and did our own context/prompt engineering. Today, there are more and more abstractions and tools available that help to solve this problem “out of the box”.

Incident management assisted by AI agents

I had opportunity to chat with engineers from DataDog at their booth. One thing is crystal clear: we are 100% moving towards AI-assisted incident management, and I think that’s a great use-case for AI agents.

DataDog, for example, has been building a lot of agentic features into their observability product, and at least their engineers say that they dog-food their own product to help them resolve their incidents/bugs faster with the help of AI agents.

The future, or should I say the present, looks like this. You get an error report from Sentry/DataDog or whatever you are using, the AI agent will do the first run trying to debug the issue, looking at all relevant error logs and traces. AI agent will try to correlate that with recent changes in your codebase, looking through your git history. Agent will use all relevant info to troubleshoot and try to solve the issue itself, or will ask clarifying information from you. In best case, agent solves the issue itself in one go. At worst case, it serves you, an engineer, as an assistant, whom you can guide to help you out, by asking it to check out logs and report you the findings etc.

I suggest you explore, experiment and adopt AI-assisted incident management at your workplace.

Docker with MCP Toolkit

Docker claims that with its MCP toolkit it’s very easy to host MCP servers locally. Probably for production use-cases as well (I don’t know). I haven’t delved into details yet, but if you have more experience dealing with Docker MCP toolkit and have opinions/thoughts, feel free to share in the comments.

Sentry: using agent inside their MCP server

Just an interesting pattern to report. Sentry exposes its MCP server. Inside its MCP server, they don’t just wrap API calls, but they have an agent that orchestrates and calls different APIs and then gives the final response.

Voice AI got crazy good! Kyutai blew my mind

The last presentation of the conference was by Kyutai. And it really blew my mind. They solved full duplex voice to voice conversation, handling interruptions natively, excellent text to speech and speech to text, seamless switching between languages, and fully recognizing all actors who speak. The presentation is here on YouTube. I strongly recommend you to watch, it’s fascinating!

And that’s a wrap!

Well, here are my random thoughts from the conference. I hope you took something useful away. One thing is clear: it’s a wild west with AI agents. so stay humble to keep learning, and at the same time be bold and curious! For more perspective on how AI agents change engineering, I suggest you to check Kent Beck’s article called Programming Deflation.

Also, we are hiring software engineers to Pactum. Feel free to check out pactum.com/careers.

Happy to continue the conversation in comments.

Here is what we talked about:

00:00:00 - Intro

00:00:05 - Welcome & Guest Introduction

00:00:33 - Alvar's Early Exposure to Computers and Coding

00:03:38 - First Steps into the Tech Industry

00:05:05 - Growth and Learning at Hansa Bank (now Swedbank)

00:06:13 - The Codeborn Experience and the Shift to Product Focus

00:08:03 - Joining the Startup World with TransferWise

00:10:00 - Evolution of Roles and Responsibilities at Wise

00:12:16 - Founding Grünfin: Mission and Challenges

00:13:20 - Comparing Different Work Environments: Bank, Contracting, Startup

00:19:51 - Key Skills for Startup Engineers

00:24:45 - Ensuring Quality in a FinTech Startup

00:35:24 - The Role of Test-Driven Development (TDD)

00:41:49 - Philosophy on Engineering Leadership

00:47:15 - Building a Team: The Importance of Composition

00:50:50 - Lessons Learned from the Grünfin Journey

00:56:57 - Why Alvar Joined Lightyear

00:59:58 - Engineering Challenges at Lightyear

01:02:44 - The Future of Engineering with LLMs

01:11:53 - Recommended Reading and Resources

00:00:05 - Intro
00:00:35 - Janne's introduction
00:01:12 - Janne's career start and moving to UX
00:03:37 - Pivotal moments in Janne's career
00:06:40 - Misconceptions about design and UX
00:12:20 - How should engineers, designers, and PMs collaborate?
00:21:54 - Working effectively with designers
00:28:42 - What other UX misunderstandings do people have?
00:31:07 - How does a company align on what is "good"?
00:44:18 - How to align on design principles and values in a growing startup?
00:53:52 - How to make a good UX in a product?
01:00:42 - How to become better at UX and design?
01:07:02 - How will design and UX evolve in the next 10 years?

And if you want to work with Janne and me, we are hiring in Pactum - https://pactum.com/careers

For more episodes, feel free to subscribe here in YouTube, or check MapForEngineers.com, thank you!

Oh, and if you want to work together with Carl and me, we are hiring in Pactum, feel free to check out https://pactum.com/careers/

00:00:00 Introduction & Current Coding Workflow
00:10:40 Junior vs Senior Engineers with AI
00:27:21 Spec-Driven Development & Tasks
00:46:31 CLI Agents vs IDE Tools (Cursor)
00:52:13 MCP vs CLI Tools & Composability
00:57:20 Claude Code Features: Slash Commands & Hooks
01:12:32 Making Codebases Agent-Native
01:31:40 AI Adoption in Organizations
01:44:16 CI/CD Pipelines & AI Automation

Thanks for listening!

Available as video on YouTube as well:

00:00:00 - Start

00:01:01 - Dmytro's background

00:13:50 - Early days of finance team in Wise

00:19:36 - Becoming lead in the finance team

00:21:19 - Squad o fteams, tribe of squads: splitting finance into subdomains

00:27:42 - Conway law: team splits reflecting architecture

00:32:56 - Event-driven architecture

00:34:51 - Context on what finance team is doing

00:36:06 - Batch approach vs real-time financials

00:39:56 - Error handling, dead letter queues

00:41:44 - Observability: Prometheus, Grafana, in-house tools

00:49:32 - Assembling the right team: finding people who care

00:52:42 - DB Skills. Postgres. Modularization. Couplings

00:57:24 - Staff+ engineering path

01:03:32 - Engineering principles in finance. Compliance, auditing

01:09:22 - Learning domain where you work

01:11:31 - Testing in finance

01:27:00 - Growing in your career: find your path, learn what interests you

Here are annotated chapters of the discussion, for your convenience:

00:00:00 Intro

00:00:00 Sneak peek

00:00:49 Episode overview

00:04:28 Anton's intro, background

00:06:37 Anton founded Codeborne: TDD and pair programming: following extreme programming principles

00:08:57 Agile is about short feedback loops

00:12:09 Under-engineering vs over-engineering

00:15:29 Tech debt and testing: engineers don't handle tech debt well enough

00:17:45 Lack of refactoring is a big problem

00:18:14 Problems with pull requests

00:27:00 Problems with squash merge

00:27:30 Good commit messages are essential

00:31:09 Good code is easy to change

00:34:34 Pair programming is continuous code review

00:36:11 Daily code review with a whole team

00:48:44 Microservices: be careful

00:59:23 Book recommendations from Anton

01:00:38 Wrap up

Full video version with annotated chapters on Map for Engineers Youtube channel

Lauri Koobas, ex-Microsoft and currently Head of Data Platform at Bondora, shed insights on data engineering - from early startup to scaling.

We mostly focused on analytics and building data warehouse - real-world challenges from both data engineering and software engineering sides. We also discussed GDPR and PII challenges when dealing with data.

Annotated chapters in timeline:

00:00:00 Sneak peek of episode

00:01:21 Episode overview

00:02:44 Introduction, Lauri's background

00:20:48 Starship robots: huge amount of data there

00:23:37 Data lake, data warehouse, data lakehouse

00:26:44 Devil is in the details: timestamps, texts, character sets...

00:49:44 Moving data from prod to data warehouse

00:53:09 Analytics tools: PostHog, Amplitude, Redash, Databricks

01:00:15 Analytics tools vs real-time monitoring like Prometheus/Grafana

01:04:15 Usability matters: each tool for its job

01:06:38 Startup grows: needs in data analytics

01:11:09 Multiple data sources: when data warehouse really begins

01:19:55 Data and (de-)coupling: software engineers should not be blocked by analytics

01:22:51 Data ETL

01:24:59 Changes in data model: multi-phase migrations

01:29:38 Change data capture, incremental imports

01:34:21 Should analytics have new data in real time? Maybe not?

01:39:02 Importing data into DWH through business events

01:43:37 When DWH subscribes to business events, data model can evolve freely

01:47:16 Quick recap what we discussed so far

01:52:25 GDPR and Data Compliance: start early

01:56:05 PII data: know exactly where you store it, control it well

02:03:37 Lauri's books recommendations on data engineering - Kimball

02:07:18 Lauri's podcast on data engineering, in Estonian

02:08:28 Wrap up

Full video version with annotated chapters on Map for Engineers Youtube channel

Annotated chapters in timeline on topics that Carl and I covered:

00:00:00 - Start

00:04:05 - Small talk, getting into the groove

00:08:18 - Carl's background: ex-Pipedrive, now engineer in Pactum

00:19:44 - Early tools: simple autocomplete and simple prompting without context

00:27:11 - AI tools with context: Cursor IDE, Continue.dev

00:43:35 - Cursor IDE Composer - Prompt+Apply to Code Instantly

00:47:29 - Ollama - following docker philosophy

00:55:15 - V0.dev - LLM to create frontend components

00:59:27 - Cursor IDE + v0.dev combination as a workflow

01:02:23 - Claude 3.5 Sonnet

01:03:10 - OpenAI o1

01:04:56 - LLMs vs SQL Queries - still to be solved

01:08:17 - LLM in TDD and Testing Workflows

01:16:04 - Focus on engineering fundamentals - LLM does not replace your engineering fundamental knowledge

01:20:13 - Book recommendations

01:29:09 - Hosting models yourself - expensive

01:33:27 - Fine-tuning models

01:35:33 - RAG

01:49:50 - Chain of thought

01:51:46 - vyce.app - GenAI helping with compliance questions

01:54:25 - Summary of tools we covered so far

01:58:43 - GenAI vs engineering careers

02:05:39 - Wrap up with Carl

Full video version with annotated chapters on Map for Engineers Youtube channel

I had a pleasure to chat with my friend Joel Mislav Kunst, who is Engineering Manager at Microsoft. We talked about growth in engineering. Some of the topics that we touched upon, with timeline timestamps are:

00:00:00 - Glimpse of episode
00:01:16 - Quick episode overview
00:05:18 - Joel's journey in software engineering
00:14:55 - Focus on engineering fundamentals
00:23:34 - Learning from all hard experiences: solving root causes
00:31:05 - Relationship between engineer and their manager
00:53:03 - Manager is not your punching bag for complaining: propose initiatives instead, be active
01:10:55 - Engineer vs manager: fork in career
01:29:17 - Generative AI in context of engineering career/growth
01:36:01 - Teaching is essential for growth. Seniors teaching juniors. Knowledge sharing
01:43:14 - Wrap up

Ürgo and I had a great chat! I will give a short summary of key takeaways that I got from this valuable discussion with Ürgo.

Map For Engineers Podcast on Youtube.

Subscribe now

Key takeaways

Work in agency is different from work in a product company

Work in agency (outsource) vs product company is different for an engineer. In agency, you get a lot of experience with different projects, but you don’t own a product or its outcomes.

In contract, in a product company, engineers should care about the end product, getting the feedback from users and customers. Responsibilities and impact are on the next level for an engineer in a product company, that you don’t get in a project-based work in an agency.

Knowledge sharing inside a team. Avoid knowledge silos

It’s important to avoid pockets of knowledge in a team, where sub-groups form, because then it’s hard to have a cohesive team. There are many tools to avoid it, for example:

• Pair programming

• Deliberately rotating knowledge among people

• Working in pairs (not necessarily pair programming, but just solving some problem together) on a topic for a short time (e.g. a week or two). But then switching pairs and topics, to avoid knowledge silos.

• I wrote in detail about some practices that I employ in Pactum to avoid knowledge silos in the team in the article Values, Principles and Practices in Engineering Team.

Product engineering begins where the comfort of the coding ends

Ürgo wrote amazing article about this topic, called Product Engineer, available in his Medium.

This diagram below is from Ürgo’s post, but it summarizes the essence of product engineering very well:

Product engineers need to establish frequent feedback loops to get signal from users on the usefulness of what they delivered. This is essential for closing the feedback loop. Once you get learnings, you repeat the process: Learn → Build → Ship → Learn → …[repeat]

Domain-Driven Design: Metaphors are important

Coming up with metaphors when modelling software is very important. When you come up with a good metaphor, try to embed it into your ubiquitous language.

GenAI in Software Engineering

GenAI won’t replace product engineers for a while. In fact, product engineering becomes even more essential than just coding. Coding is just a tool, a means to an end. Product engineering skills will be ever so valuable - to understand which product to build, to iterate, to learn from your users and customers, to be creative. Product engineers will leverage GenAI tools to automate non-interesting tasks (e.g. creating this next frontend component, if it can be automated quickly).

And that’s a wrap! I will be recording new episodes soon. Feel free to subscribe if you found it valuable.

Subscribe now

Map for Engineers Podcast is also available on Substack, Spotify, Apple Podcasts, or on any of your favourite podcast app.

See you in the next episode!

Vitalii Lakusta

Have you ever had a moment where you were part of a critical and important meeting, only to see people nodding and agreeing with each other? I can only guess that you probably could feel at your core that something is wrong, but maybe it was hard to point the finger on it. Maybe you thought at that moment, “What’s really going on? Why are we not speaking out our minds and opinions boldly? Why aren’t we having discussions over real issues here?” Over time, a team might get used to this and it could become a team’s new norm, if the team is complacent and doesn’t actively do anything about it.

I believe it shouldn’t be this way. First of all, if you are a team lead, you are responsible for promoting a culture of critical thinking in the team. You can promote it by saying something like, “I might have blind spots here reasoning about this problem…please help me here with your original and fresh perspectives”, or, for example, “Let’s all think independently for a few seconds…we need all our brains on this problem and think actively. There are many possible solutions to this problem, and it’s not yet obvious which one to pick…” I could bring more examples, but I hope you get the idea.

But, I believe this can be promoted and supported by all team members. All team members can be leaders and can help its team to grow by showing an example. And if a team member wants a team to grow into applying more critical thinking and avoiding groupthink, this member can lead the way. As long as your team is aligned on its values, principles, and practices, everyone can lead and support the team to move in the right direction!

So, next time you go into that meeting…start from yourself first. Observe and reflect. Do I think critically and independently about the problem here? What do I think about this problem? How would I solve it? What are my opinions on this?

Applying critical thinking and thinking independently doesn’t equal arrogance. It’s a very good idea (I think) to build on top of each other ideas during a meeting, to synergize ideas. But while doing so, think. And listen actively at the same time.

So, go ahead - have good and respectful “creative conflicts” and different ideas floating in the air in the next meeting - it’s healthy - essential even - to build a strong team. Paradoxically, the diversity of ideas and critical thinking of all team members will help you to forge the psychological safety in your team, which is essential to become a highly-performing team.

This post is meant for reflection. We all already know about things that I wrote above…but sometimes we forget about it and it’s good to remind ourselves and reflect again. So this post is a reminder and call for reflection - to myself, and maybe to you, about the importance of thinking independently and avoiding groupthink in your team.