\ud83d\udee0\ufe0f Debugging Story<\/p>\n\n\n\n
I finally got my mini AI-RAG and chatbot service talking to my app. Still a prototype \u2014 text only, nothing fancy to look at. But when it all clicked together, that feeling made every late night worth it.<\/p>\n\n\n\n
\u26a1 FastAPI\ud83d\udc33 Docker\ud83e\udd16 Ollama\u2601\ufe0f Cloud Run<\/p>\n\n\n\n
\ud83c\udfd7\ufe0f Service Architecture<\/p>\n\n\n\n
\ud83d\udcf1 App\u2192\u2601\ufe0f Cloud Run\u2192\ud83d\udd12 Cloudflare\u2192\ud83c\udfe0 Home Router\u2192\ud83d\udda5\ufe0f Prod Server\u2192\ud83d\udcbb Dev Server (RTX 3060)<\/p>\n\n\n\n
\ud83d\ude30As the layers stacked up, strange things started happening<\/p>\n\n\n\n
iptables + NordVPN + Docker were interfering with each other \u2014 packets were vanishing, and tcpdump was the only thing giving me any direction. My MacBook’s SSH tunnel kept dropping and reconnecting, and at some point the dev server started silently swallowing requests. Nothing in the logs, server still running. Rebooted the MacBook. Fixed immediately.<\/p>\n\n\n\n
Once I got through all that, a 404 was waiting for me.<\/p>\n\n\n\n
\ud83d\udea8The route exists. So why 404?<\/p>\n\n\n\n
\ud83d\udca1 From inside the container (127.0.0.1): 401 Unauthorized<\/strong>. From outside via nginx: 404 Not Found<\/strong>. Same container, same process, different results.<\/p>\n\n\n\n \ud83d\udd0eThe key clue: response size<\/p>\n\n\n\n The response sizes in the nginx access log looked wrong.<\/p>\n\n\n\n \u26a0\ufe0f FastAPI default 404 Working backwards, it was an Ollama error message:<\/p>\n\n\n\n {“detail”:”model ‘llama3’ not found, try pulling it first”}<\/p>\n\n\n\n \ud83d\udd75\ufe0fRoot cause: upstream error propagation<\/p>\n\n\n\n A client service was reading \ud83e\udd16Debugging with AI<\/p>\n\n\n\n I had been relying on AI to analyze the flood of logs. At some point, it started nudging me toward tearing down the architecture itself. I stepped back, worked through it with Claude from scratch, and eventually tracked down the real cause.<\/p>\n\n\n\n AI compressed what could have been weeks of debugging into days. But holding the design together was still a human job. The more network layers you add, the easier it is for AI to lose the thread too.<\/p>\n\n\n\n \ud83d\udcddWhat I took away<\/p>\n\n\n\n 1\ufe0f\u20e3<\/p>\n\n\n\n Check the response size<\/p>\n\n\n\n 22 bytes vs 64 bytes. Two 404s can mean completely different things. That single number was the decisive clue.<\/p>\n\n\n\n 2\ufe0f\u20e3<\/p>\n\n\n\n 404 doesn’t always mean routing failure<\/p>\n\n\n\n Upstream errors propagate silently. If you don’t handle them explicitly, they’re very hard to trace.<\/p>\n\n\n\n 3\ufe0f\u20e3<\/p>\n\n\n\n Wrap upstream errors as 502\/503<\/p>\n\n\n\n Returning internal errors and routing errors with the same status code multiplies your debugging time significantly.<\/p>\n\n\n\n 4\ufe0f\u20e3<\/p>\n\n\n\n When nothing makes sense \u2014 reboot. Seriously.<\/p>\n\n\n\n Sometimes the most powerful debugging tool is turning it off and back on. Your mental health will thank you.<\/p>\n\n\n\n When the logs go quiet, widen your view. \ud83d\udd2d <\/p>\n","protected":false},"excerpt":{"rendered":" \ud83d\udee0\ufe0f Debugging Story The Route Exists. So Why Is It Returning 404?A FastAPI + Ollama Deep Dive \ud83d\udd0d I finally got my mini AI-RAG and chatbot service talking to my app. Still a prototype \u2014 text only, nothing fancy to look at. But when it all clicked together, that feeling made every late night worth […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[12],"tags":[7,3,9,5,4,6,8,10],"class_list":["post-9","post","type-post","status-publish","format-standard","hentry","category-it","tag-ai","tag-backend","tag-debugging","tag-docker","tag-fastapi","tag-ollama","tag-rag","tag-softwareengineering"],"_links":{"self":[{"href":"https:\/\/blog-api.minpox.com\/index.php?rest_route=\/wp\/v2\/posts\/9","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog-api.minpox.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog-api.minpox.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog-api.minpox.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog-api.minpox.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=9"}],"version-history":[{"count":1,"href":"https:\/\/blog-api.minpox.com\/index.php?rest_route=\/wp\/v2\/posts\/9\/revisions"}],"predecessor-version":[{"id":10,"href":"https:\/\/blog-api.minpox.com\/index.php?rest_route=\/wp\/v2\/posts\/9\/revisions\/10"}],"wp:attachment":[{"href":"https:\/\/blog-api.minpox.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=9"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog-api.minpox.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=9"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog-api.minpox.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=9"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}POST \/internal\/llm\/chat<\/code> kept returning 404 inside Docker while working perfectly on the dev server. I printed the route table directly \u2014 the endpoint was clearly registered. It wasn’t a proxy issue either, since requests were showing up in the FastAPI logs.<\/p>\n\n\n\n{\"detail\":\"Not Found\"}<\/code> = 22 bytes<\/strong>
Actual response sizes = 64\u201389 bytes<\/strong>
Something else was generating that 404.<\/p>\n\n\n\nOLLAMA_MODEL=llama3<\/code> from GCP Secret Manager and passing it in the request body. The model actually installed was llama3.1:8b<\/code>. Ollama’s 404 was propagating straight through FastAPI to the client \u2014 making it look like the route didn’t exist. Not a routing problem. An upstream error propagation problem.<\/p>\n\n\n\n
\n\n\n\n
And when nothing makes sense \u2014 reboot first.<\/p>\n\n\n\n