We Tested DeepSeek V4.1 Flash on Our Own Chat-O Codebase, Then Gave It Our Photo Editor

Published: 2026-09-15

The result was promising and instructive. The upgrade was clean, the full automated suite passed, and the model successfully worked across backend state, LiveView UI, image-processing code, and focused tests in the studio. Human and automated review still found edge cases worth fixing. That combination is the useful story: what the model accomplished, how we verified it, and where review still mattered.

Chat-O is an Elixir and Phoenix LiveView app. We upgraded Phoenix LiveView, which renders the interface, and salad_ui, the generator used for components that are vendored into the app. The upgrades were coupled: salad_ui 0.14.9 did not parse under the LiveView 1.2 tokenizer, which was the reason LiveView had been pinned to 1.1.x.

That made this a useful agent benchmark. The code change was small, but getting it right required dependency reasoning, production-build awareness, automated tests, and browser QA. We ran the task with DeepSeek V4.1 Flash through OpenCode Go in a single session.

The change

The dependency upgrade itself changed two files, with 17 insertions and 17 deletions:

 mix.exs  | 16 ++++++++--------
 mix.lock | 18 +++++++++---------
 2 files changed, 17 insertions(+), 17 deletions(-)

The meaningful part of mix.exs:

-      {:phoenix_live_view, "~> 1.1.0"},
+      {:phoenix_live_view, "~> 1.2"},

-      {:salad_ui, "~> 0.14"},
+      {:salad_ui, "~> 1.0", only: [:dev]},
+      {:tw_merge, "~> 0.1"},

-      {:igniter, "~> 0.5", only: [:dev]},
+      {:igniter, "~> 0.6", only: [:dev]},

The line worth highlighting is salad_ui. Version 1.0 depends on igniter, a code-generation toolkit. Chat-O does not call salad_ui at runtime because its generated components are vendored and use TwMerge directly. The agent therefore kept salad_ui and igniter development-only, then added tw_merge as a direct dependency.

That is more than dependency-file bookkeeping. It keeps salad_ui, igniter, and the rest of the code-generation toolchain out of the production dependency set while retaining the one runtime library the vendored components need.

Verification

Tests were the baseline, not the finish:

Development compilation succeeded.
Production compilation succeeded, with salad_ui and igniter absent from the production build and tw_merge present.
100% of the automated test suite passed.

Then it did the part we did not ask for: a manual QA pass in a real browser.

Flow	Result
Login and session	works, socket connected
Chat with image history	renders, image messages intact
Image Studio modal	opens; thumbnails; main image loads; edit box present
Thumbnail click	switches versions
Arrow-key navigation	moves between versions, with wrap
Escape	closes the modal
Sidebar tabs	panel switches
Prompts panel	opens with the template list
Room picker	shows the available personas
Composer	accepts and holds typed text
Settings	renders, connected
Blog post	images load, related posts present

The checked flows produced no console errors, broken images, or reconnect banners.

OpenCode Go running DeepSeek V4.1 Flash: the verification table and run stats, 10.5s at 174.5 tokens per second

The screenshot reports 10.5 seconds and about 174 tokens per second for the displayed model turn. That is a single observed turn in our setup, not the wall-clock duration of the full upgrade, compilation, test, and browser-QA session. It is useful as a glimpse of generation speed, not as a standalone end-to-end benchmark.

Why the manual pass mattered

The full automated suite passed, but it does not drive every browser interaction or render every state of every vendored component. In particular, it does not open the Image Studio modal and use its keyboard navigation. That is exactly the kind of surface a UI framework upgrade can break while an otherwise healthy suite stays green.

The browser pass complemented the automated suite. It opened the app, exercised the named flows in the table, and checked observable browser behavior: image loading, console output, and LiveView connectivity. That gave us useful coverage of several interactions the existing suite does not drive.

This distinction matters. Automated tests, targeted regression tests, and browser walkthroughs catch different classes of problems. A good agent run should use them together and report the boundaries of each, not treat one as a substitute for the others.

It even had to unstick its own tooling. A previous automated browser was holding a profile lock and blocking a fresh launch, so the agent found the process, cleared it, and relaunched.

Then we gave it the Photo Editing Studio

After the upgrade benchmark, DeepSeek V4.1 Flash worked on a more demanding, separate change: inline editing and visual annotations in Chat-O’s Photo Editing Studio.

This was not another dependency bump. The task crossed several layers of the application. The model worked on:

rectangle, circle, and freehand annotations over a source photo;
sending both the marked-up image and the editing instruction to the image model;
pending, gallery-navigation, and Before/After viewer states;
preserving the relationship between an edited image and its source across reloads;
extracting viewer transitions into a testable state module; and
focused tests for the renderer, viewer state, LiveView events, and UI rendering.

All of the focused tests we ran against that follow-up work passed. More importantly, deeper review found cases the happy path did not expose: failed jobs needed to release the pending state, malformed or oversized drawing data needed stronger limits, temporary image files needed guaranteed cleanup, and some image preparation work should move out of the LiveView process.

That does not make the experiment a failure. It makes it representative of real agent-assisted development. DeepSeek produced a substantial working feature, responded to review by improving state handling and test coverage, and left a smaller, concrete set of reliability issues for another pass. “The demo works” and “the change is ready to merge” are different milestones, and a useful benchmark should show both.

For the complete feature timeline, including the browser crash, product feedback, test work, and review fixes, read how DeepSeek V4.1 Flash built draw-to-edit for Chat-O’s Photo Studio.

Where this lands

The dependency upgrade looks clean. The dependency decision is technically sound, the existing suite passed, the production build excluded the development-only tooling as intended, and the browser pass covered several high-value interactions outside that suite.

The Photo Editing Studio follow-up raised the difficulty and made the lesson clearer. The interesting part was not how many files the model changed. It was that DeepSeek could reason across dependencies, production boundaries, interactive UI state, image processing, persistence, and tests in a real codebase. It also showed why we still review agent work like any other change before it goes to production.

We are also adding DeepSeek V4.1 Flash to both Chat-O’s Balanced and Power auto-selection pools. At the time of writing, OpenRouter lists it at $0.15 per million input tokens and $0.60 per million output tokens, with a context window of roughly one million tokens. That is inexpensive enough for Balanced while capable enough to be one of the options Chat-O can select for harder Power-tier requests.

For more of our agent-driven engineering, see how we migrated a full-stack codebase from Rails to Go with a model most people had not heard of.