Should I Share My Prompt And Not My Code?

Share
Should I Share My Prompt And Not My Code?

This is a story about how writing a text formatter gave me a crisis of value.

Whenever I read with my e-reader, I highlight sections and make annotations. I can export these to my notes app to make them easier to find later, but the apps don't play nice with each other - I have to reformat the text and fix a bunch of encoding errors.

Luckily, this is totally solvable with a quick Python script to convert formats. It's a mess, and there's no way I could support it if I gave it someone else, but it's fine if I'm the only one who will ever use it. It's like cooking for myself - sometimes I have girl dinner, but I would never serve it a dinner party.

So I've now got tidy little tool that handles my niche use case and I can get back to reading.

Wait, is it even worth publishing code like this to GitHub?

My whole career I've been publishing everything I write that's not contractually restricted. I've talked and written about the value that comes from working in public, and I stand by this for large projects looking to build a community. I'm less sure about it for these "duct tape" tools that solve a specific problem in a specific context.

Me in 2023, trying to get companies to just publish stuff

I used to feel like even my most niche tools were worth putting out in the world, because it:

  1. Showed people what kinds of projects I work on, and
  2. There was always a chance someone else had the same problem and could adapt it for their own purposes.

But today if I need to write a small connector piece of software I usually do it with a coding agent - it's faster and I'm not that worried about the ramifications of AI Slop when it's just a little utility.[1]

But here the argument for publishing everything starts to break down:

  1. I didn't write the code manually, so you can't learn about my style from it.[2]
  2. Everyone else is using agents anyway, so they are more likely to prompt a solution than find my zero-star repo on GitHub.

In fact, given my prompt - or several prompts assembled into a product spec - you could get an agent to write something functionally identical. All the value I generated, really, was in defining the intended use case. As Spec-Driven Development advocates point out, the most direct artifact of that work is the product spec.

Is it possible, then, that we should all just be publishing product specs instead of code?[3] If we imagine that a product spec is like a compressed natural-language version of a codebase, then why not use an agent to unpack that version into functioning code?

What if the product spec is the compressed codebase?

It has an appeal. Because the spec is written in natural language, it's easier to understand its purpose. It's also easier to adapt to a new context - tweaking one requirement is much faster than updating a data class, its dependencies, and relevant tests.

You lose a lot of the guarentees code gives you, but maybe that doesn't matter?

Yes, I do think this is relevant to lab automation

The primary code we write in lab automation is integration code - small tools that connect a lab execution system to a liquid handler, or a plate reader to data platform. Everyone writes this code, but it's usually not worth sharing because it's highly dependent on lab context.

But maybe it is worth sharing the specifications. Then the agent just needs to merge in your lab's context when it "unpacks" the code.

Let's test the idea

As an experiment, I took my little note parsing utility and had an LLM summarize it into a functional spec, which I reviewed and thought was reasonably accurate.[4]

I then copied its output into a file called SPEC.md in a completely new clean repo and told it to make the app described there. It actually seemed to be humming along well, until:

imgs_app_BioApplications_atZFijwhoe.png

Well shoot, I used my whole token allotment. This is a tiny utility! I'm not paying spot pricing to finish this.

One week later

My token budget reset and the agent finished up. The resulting tool worked, although with some minor differences in output. These would break a strict format checker, but really are totally fine for my purposes. So I think it's fair to say it matches the design intent.[5]

I can quibble with the architecture and maintainability (174 tests??), but really my big concern is that token limit I hit. This is such an inefficient and costly way to share code that I'm not sure it still counts as sharing.

To revisit my dinner party metaphor from earlier, it feels like if I invited you over for a dinner party and, instead of serving you piping hot food, I gave you a recipe card and told you that you can buy food at Safeway. Sure, recipes are nice, but it wouldn't really be good hosting, would it? Doesn't feel like a good way to build community.

So should I share my prompt and not my code? I'm not sure.

I think there's a dark future where the only way my code gets read is as part of LLM training and the only code anyone uses is generated by their AI agents. In that case it doesn't matter. But personally, I hope we can keep building community around our work, so I'm going to keep sharing my code - I'll just commit my spec as well.


  1. But oh boy to I worry about it for large production systems! ↩︎

  2. Your mileage may vary depending on where on the "vibe coding" spectrum you fall. I'm opinionated about architecture even if the actual code is auto-generated, so I guess there is a little bit of me showing up in the final product. ↩︎

  3. For small projects specifically. SDD best-practice includes a number of documents and prescribed workflow for more complex systems, and I don't think it would be as easy as a one-prompt "unpack". ↩︎

  4. Truthfully, it was a remarkably dry read, and I'm concerned that that's already enough friction that most people would prefer the vibe-coding experience to reading and adapting this spec. I think if the spec is to be precise you have to enumerate the rules like this, but it might be a case of A couple of months in the laboratory can frequently save a couple of hours in the library. ↩︎

  5. Still, I'd carefully examine the results if I was implementing a lab automation system with strong schema requirements. ↩︎