Default podman compose up configuration to a known model with cached data and demo mode enabled.

[John Matthews]

I think it may help end users who are evaluating Kai if we make the configuration as simple as we can for an easy evaluation.

For this reason I'm proposing we:

• Enable DEMO_MODE to True by default in compose.yaml
• Ensure we have valid cached data for running docs/scenarios/demo.md for the default model

The goal is to allow an end user to run podman compose up without modifying any env vars or kai/config.toml and then be able to use cached data for a quick evaluation of Kai.

[Maira Usman]

Hey @jwmatthews , Do you mean this file docs/scenarios/demo.md? As the link mentioned above is not found.

[Jonah Sussman]

I'm not too sure about having KAI__DEMO_MODE enabled by default. Enabling demo mode is fairly trivial (KAI__DEMO_MODE="true" podman compose up). This way, it's explicit that they're using cached data, and could avoid potential confusion if they want to generate solutions via their LLMs.

[John Matthews]

I'm not too sure about having KAI__DEMO_MODE enabled by default. Enabling demo mode is fairly trivial (KAI__DEMO_MODE="true" podman compose up). This way, it's explicit that they're using cached data, and could avoid potential confusion if they want to generate solutions via their LLMs.

I am concerned that while this is easy it is another data point of something to document/know and does add a potential friction when a user 'forgets' or doesn't see and runs without explicitly enabling demo mode. I believe this is against how we should think for the expected persona to use podman compose up.

Please consider this from the expected user perspective of a new individual unfamiliar with Kai, first time they come in to see what it can do. They are not yet sure if Kai is worth their time/attention...they want/need a clear signal Kai has potential to help so its worth learning more. I would like to focus the experience of consuming podman compose up for this specific user perspective.

Then I would assume once the user has seen value in Kai, they will be motivated to read docs and explore configuration options and customize as they desire. At this point I can see them opting in to disable demo_mode if they choose to regenerate code fixes, but I think this is the less likely desire from the user running podman compose up. I think the consumer of podman compose up is not interested in nuances of regenerating fixes in real-time, I think they are primarily focused on assessing potential value of Kai so they can quickly show this value to others or decide for themselves if Kai is worth more of their attention.

Let's consider that I expect we have 2 user personas as of today.

• Evaluator: First time user of Kai, they are trying to determine if Kai provides value and is worth more time/energy to dive in
  • Deployment: podman compose up
• Contributor: They believe in Kai's vision/approach and are determined to learn/extend/use. This may be a user helping us to improve Kai via feedback or a developer improving functionality
  • Deployment: make run-server

[Jonah Sussman]

I think you're making a good point about simplifying the experience for Evaluators, but I believe there's another user group we should consider. These are users who are past the evaluation stage but aren't quite Contributors yet—they expect to use Kai in a more operational capacity, but not in demo mode.

For these users, having KAI__DEMO_MODE enabled by default might be confusing. They’d likely expect to work with live data and generate solutions with their LLMs without needing to reconfigure. While it's great to optimize for Evaluators, we might unintentionally create friction for those users who are ready to move beyond evaluation and use Kai more fully.

I believe it would be better for KAI__DEMO_MODE to be an opt-in feature rather than opt-out. This way, users are explicitly choosing to enter demo mode, which reduces the risk of confusion for those who intend to use the full capabilities of Kai from the start

[Jonah Sussman]

I am fine with leaving KAI__DEMO_MODE=true in the compose.yaml for now, as long as we document this behavior somewhere, and intend for it to be reversed at some point in the future.