Tokens, part two: the hundred-dollar memory that ran better on my own graphics card

This summer I gave my agents a shared long-term memory, because they forget everything when a conversation ends. I had a small service on my server read every page I keep, so any agent could recall the facts later. The extraction ran on a paid cloud model from one of the big names, metered per token, and the estimate for the first fill was $11, perhaps $14. The estimate held. It was very good. The first fill took under an hour, and 3 days later the memory held 26,342 facts and the agents remembered across sessions. On August 25th the writes started failing while the reads kept working. That evening the alert read 3 cents left, on a credit I had just topped up with a little over $100 because it was running low. The first diagnosis was that the memory was the victim: its visible calls came to about $7 a week, so something else on the shared credit must have eaten the rest. The next morning the archivist agent took that back. The spender was the memory after all, just not the part anyone had looked at: after every fact it stored, it went back over what it already knew, tidied, merged and summarised, and that round grew with the memory. Two thirds of the money had gone to the memory organising its own memories. The credit ran out, so I moved the whole job to the 5-year-old graphics card I already owned, and it took that card 2 more days to finish the expensive tidying. Slower, and free to run. I put the same test prompt through both models, and the paid one found 5 facts, mine found 8. So I had paid the big name extra to be the worse one. The card still runs for hours at a stretch, adding memories; the same open model in the cloud would do that for about $3.50 a month instead of about $126, but the memory stays on my own card, in my own hands. Cheap advice: try the open model before you pay for the big one.