Case studyAI Product Engineering · Personal Project2026

Open Fern

A workspace for creating video with generative AI: refining a prompt, choosing and configuring a model, tracking the generation, and reviewing the result. Built end to end by me.

Live product, Open Fern

open fern · creation workspace

Prompt

A slow aerial pass over a coastline at first light.

Refined

editable

Aerial tracking shot, low sun, long shadows across wet sand, 24fps, shallow depth of field.

Capabilities resolved per model. The UI never branches on provider.

Configuration

Model

Duration

Aspect ratio

Reference image

Jobs

Scene 01Complete
Scene 02Generating
Scene 03Needs review

The creation workspacePrompt and its refinement on the left, model configuration on the right, and generation jobs tracked underneath. Three things that are usually three separate tools.

Interface recreated for portfolio purposes.

Problem to outcome

The complete case study in seven parts. The sections below provide the engineering detail behind this summary.

Problem

Generative media is fragmented across models, providers, prompts and assets. The tools that exist are either a thin wrapper around one provider's settings, or an abstraction so general that each model's actual strengths disappear.

Context

A workspace for creating video with generative AI: refining a prompt, choosing and configuring a model, tracking the generation, and reviewing the result. Built end to end by me.

Constraints

Different models behave differently
Generation is not instant
Failure is normal, not exceptional

My role

End-to-end ownership across product thinking, architecture, interface design, and implementation—treating AI as part of the product rather than an isolated API integration.

Approach

When someone starts a generation, the product translates the chosen model and its settings into a request that particular provider understands, while the interface keeps tracking one consistent job. That is what lets the experience stay the same regardless of which model was picked.

Solution

One workspace where the whole loop (configure a model, refine a prompt, generate, track, review, keep) behaves like a single product rather than a chain of separate tools.

Outcome

Open Fern is live and in active development. The part I would point at is the model description system: adding a provider is a description and an adapter, and the creation interface does not change.

: providers behind one creation interface
: media types in one loop: text, voice, video, composition
: provider names in the creation UI, by construction
: person: product thinking, architecture, design and code

What is Open Fern?

Open Fern is a workspace for creating video with generative AI. It covers the whole loop: turning an idea into a prompt, choosing and configuring a model, tracking the generation, reviewing the output, and keeping the assets and brand context that make the next attempt better than the last.

A single video pulls on several kinds of model at once (text for scripting and prompt refinement, voice for narration, video for the footage) and then has to assemble the results into something coherent. Current integrations span OpenAI, Claude and Gemini for text, ElevenLabs for voice, Kling for video, and Remotion for composition and editing.

The creation loop

01
Choose what to create
Scene, narration, full sequence
02
Select and configure a model
Only the controls it supports
03
Write and refine the prompt
Refinement is visible and editable
04
Generate
Accepted as a job, not awaited as a request
05
Track the generation
Real stages, survives a closed tab
06
Review the output
An intermediate result, not an asset
07
Refine or regenerate
Per scene, not the whole project
08
Save to assets
Only what the creator chooses to keep

Every step is a place the product has to make a decision on the user's behalf, or deliberately refuse to.

What it does

Creation workflows: Generate different forms of media (script, narration, footage, composition) from one experience instead of four tools.
Model configuration: Expose what a model can actually do without requiring the user to learn each provider's API vocabulary.
Prompt refinement: Move an idea toward a usable generation request, showing the rewritten prompt rather than silently substituting it.
Generation tracking: Represent queued, processing, completed and failed work clearly, and keep it visible without blocking the workspace.
Asset management: Keep outputs organised across generations, versions and revisions instead of treating every generation as disposable.
Human review: Keep the creator in control of choosing, refining and approving what becomes part of the final piece.

Why the problem was difficult

Calling a generation API is a POST request. Building a product around generation is a different problem, and most of it has nothing to do with the model.

A creator does not experience a request and a response. They experience an intention, a configuration they half understand, a wait of unknown length, a result they have to judge, and a decision about whether to keep it or try again. Every one of those steps is interface work.

So the product has to hold model differences, configuration, asynchronous jobs, progress, failure, retries, several competing generations, output history, asset organisation, and, unavoidably, human judgement about whether the thing that came back is any good.

How do you hide the complexity of several AI systems without hiding the control people actually need?
The question the rest of this page answers

Contribution and ownership

Product

What should exist, and who decides?

Which workflows the product supports
What a creator needs to see at each step
Where AI assists and where the user stays in control
The distinction between a generation attempt and a kept asset

UX

How should it behave?

How generation states are represented over minutes, not seconds
How model configuration stays honest without becoming a settings dump
How outputs are compared, refined and chosen
How errors and retries behave without becoming dead ends

Frontend

How is it built?

Component architecture and feature boundaries
Client state, server state and long-running job state
Creation flows and configuration surfaces
Asset library and review experiences
Responsive behaviour and media performance

AI integration

How does the model reach the product?

The model description system that drives provider configuration
Prompt refinement workflows
Generation lifecycle, from accepted job to reviewed output
Failure, timeout and retry handling

What isn’t mine

OpenAI, Claude, and Gemini handle text; ElevenLabs handles voice; Kling handles video; and Remotion handles composition. The contribution covered the product around these models, not their training, fine-tuning, or hosting.

The constraints shaped the product

Different models behave differently

Providers disagree about inputs, parameters, capabilities and output formats. Two video models will not accept the same request, and the differences are not cosmetic.

The interface could not branch per provider without becoming several interfaces sharing a stylesheet.

Generation is not instant

Work takes seconds to minutes, and is accepted long before it is finished. Request → spinner → response is not a model of what is happening.

Generation had to be a stored job with observable stages, not a loading boolean.

Failure is normal, not exceptional

Providers fail, jobs time out, and a generation can succeed technically while producing something unusable.

Retry and regeneration are part of the main path, not an error branch bolted on afterwards.

More control means more complexity

Exposing every model parameter makes the product powerful and intimidating at the same time.

Common controls surface by default; model-specific ones are revealed progressively.

Every generation must stay explainable

Weeks later it should still be possible to answer what prompt, what refinement, what settings and what model produced a given output.

Generation history is stored as a first-class record, separate from the assets a creator chose to keep.

How it fits together

Creator
The person making something
Product UI
CreationJob stateAsset state
AI / API layer
Model descriptionProvider adapters
Providers
OpenAIClaudeGeminiElevenLabsKling
Generated output
Returned, stored, unreviewed
Review
A person decides
Asset library
Only what was kept

One product surface fans out to several providers and converges again on a person deciding what to keep.

Decisions that shaped the product

Each of these solved a real problem and cost something in return. The trade-off is stated alongside the decision, because a decision without one was not really a decision.

Decision 01

Describing what a model can do, instead of which model it is

Problem

Video models expose completely different controls (durations, aspect ratios, seeds, reference images, motion settings) under different names with different valid ranges. Branching the interface per provider produces several interfaces sharing a stylesheet.

Decision

Describe each model by what it can do, and build the configuration screen from that description. The interface asks what this model supports, never which provider it came from.

Trade-off

A shared description risks reducing everything to the lowest common denominator, so a genuinely unique feature has to be expressed as a capability or given an escape hatch. Both cost more than a one-off branch.

Result

Adding a provider is a description plus an adapter, and the creation experience stays one experience.

Decision 02

Treating a generation as a job, not a request

Problem

Video generation takes minutes. Treating it as a normal request means a closed tab loses the work, and a spinner is a lie about a process that has real, reportable stages.

Decision

Model each generation as a stored job with observable stages. The interface watches the job rather than owning it, so progress survives a reload and can be picked back up.

Trade-off

Considerably more infrastructure than firing a request and waiting, and the job state itself becomes something that can go stale or wrong. Necessary once a unit of work outlives the page.

Result

Users can close the tab, come back, and find the work where they left it.

Decision 03

Showing the rewritten prompt instead of hiding it

Problem

Rewriting the user's prompt behind the scenes makes results unpredictable and impossible to learn from. You cannot tell whether a bad result came from your idea or from the rewrite of it.

Decision

Make refinement a visible, editable step: show the rewritten prompt, let the user change or reject it, and keep it with the generation record.

Trade-off

One more step between intent and output, which is exactly what most tools remove to feel magical. Being able to tell why something worked is worth more than the magic.

Result

Results are reproducible and diagnosable, and users get better at prompting rather than luckier.

Decision 04

Keeping the video editable instead of exporting it

Problem

Review is only useful if acting on it is cheap. If the finished video is a single exported file, changing one scene means regenerating everything after it, which teaches users to skip review.

Decision

Keep the video as a composition that references its generated pieces, rendered with Remotion, rather than baking everything into one file. Regenerating a scene replaces one piece and the rest re-renders around it.

Trade-off

The composition is tied to Remotion's rendering model, so moving to another renderer would be real work. Preview and final render also have to be kept honest with each other.

Result

Rejecting a scene costs that scene. Human review stays cheap enough that people actually use it.

Designing for non-deterministic systems

Conventional software is close enough to deterministic that the interface can promise an outcome: an input goes in, a known operation runs, a predictable result comes back. Error states are the exception.

Generative systems break that promise. The same prompt produces different output twice, quality varies, and 'succeeded' does not mean 'good'. The interface has to carry that difference honestly instead of pretending it away.

Progress: The user needs to know whether work is queued, running, finished or failed, and the difference between queued and running matters when the wait is minutes.
Control: Enough configuration to influence the result, without a settings panel that requires knowing a provider's API.
Recovery: Failure cannot be a dead end. The path from a failed generation back to a running one is a product feature.
Iteration: Regeneration is the normal path, not a correction. Designing it as an error case makes the common case feel like a mistake.
History: Users need to see what they generated before, because comparison is how they decide what is good.
Human judgement: The system supports a decision rather than asserting the output is correct. Nothing becomes an asset without someone choosing it.

Frontend architecture

The boundaries that matter here are not component boundaries. They are about which kind of state owns which fact, because generation work outlives the components that started it.

UI state

Dialogs, selection, local interaction, what is expanded

Ephemeral and component-local. Never persisted, since it would be noise on reload.

Workflow state

The multi-step creation flow and its configuration

Survives navigation within a project, so a half-configured generation is not lost by clicking away.

Domain state

Generation lifecycle and job status

The one boundary that had to be explicit. A job is a server fact, not a component's loading flag.

Server state

Models, jobs, assets, generation history

Cached and revalidated. The interface reads it; it never becomes the source of truth.

Deep dive: the generation lifecycle

A generation is accepted, then happens somewhere else, then either produces something or doesn't. Modelling that as a boolean loses every state that matters to the person waiting.

UI: Status is a stage with a name, not a spinner. A running generation stays visible without blocking the rest of the workspace.
State: The job is stored server-side and rehydrated on load, so a reload, a navigation or a closed tab costs nothing.
Failure: A failed job keeps its prompt and settings, so retrying is one action rather than a re-entry of everything.
Experience: The creator can start something slow and go elsewhere in the product, which is only possible because the job does not live in component state.

Trade-offs I had to make

Flexibility vs simplicity

Model-specific controls give advanced users real power and give everyone else a wall of settings they cannot evaluate.

ChoiceExpose common controls by default and reveal model-specific options progressively, driven by what the model description says it supports.

Immediate feedback vs truthful state

A spinner is a few lines of code and communicates almost nothing about work that takes minutes and can fail halfway.

ChoiceModel the generation lifecycle explicitly and show the stage. More state to manage, and a far more honest product.

Provider abstraction vs provider capability

A fully generic interface is easier to reason about, and quietly erases the specific things that make one model worth choosing over another.

ChoiceNormalise the shared concepts and let the model description carry genuinely provider-specific capability. The right boundary remains an open product trade-off.

Where it is now

Open Fern is live and in active development. The part I would point at is the model description system: adding a provider is a description and an adapter, and the creation interface does not change.

As a portfolio piece its value is the reasoning rather than a metric. It is where I get to make the architectural decisions unilaterally, and be judged on them.

Built: A unified creation loop across text, voice, video and composition
Supported: Five providers behind one configuration interface
Designed: A generation lifecycle with real failure and retry paths
Created: Asset management separating attempts from kept work
Shipped: A live product, at openfern.com

What this experience changed

Model capability isn't product capability

A model can be genuinely impressive and still useless until someone designs the workflow, the waiting, the failure and the review around it. Most of the work in this project was that gap.

Async state is the product, not an edge case

In a generative tool, waiting, retrying and failing are the main path. Treating them as exceptional produced an interface that felt broken during ordinary use.

Abstractions need escape hatches

The first model description system anticipated providers that were not yet integrated, and the first two real integrations immediately disagreed with it. A better approach would start with two concrete integrations and extract the shared shape from actual requirements.

Human review is a feature, not a fallback

The goal was never maximum automation. Keeping the creator in the loop is what makes the output usable, and designing review as a first-class step rather than a safety net changed the structure of the product.

What I'd improve next

01Generation comparison: putting competing outputs side by side rather than in sequence
02History as a browsing surface, which is how it is actually used, rather than a status list
03Evaluation, so 'is this better than last time' is answerable with something other than memory
04Cost visibility per generation, before running it
05Asset lineage: which attempt a kept asset came from, and what produced it
06More resilient job recovery when a provider fails mid-generation

Next case study

Community Platform for 500K+ EducatorsA discussion and collaboration platform used by more than 500,000 educators across a US state, with the administration and moderation tools a public-sector deployment that size requires.All work

05 / Contact

Have a complex product problem worth solving?

Available for senior frontend, product engineering, and AI product engineering opportunities.

Start a conversation LinkedIn

connection.openavailable

01
Email
habibdeoraji@gmail.com
02
LinkedIn
View professional profile
03
Location
Bangalore, India · Remote