Hwandji commited on
Commit
6f49197
Β·
1 Parent(s): 582752d

docs: Add routing fix analysis and task context documentation

Browse files

- ROUTING_FIX_ANALYSIS.md: Complete analysis of API routing issue
- TASK_CONTEXT_API_ROUTING_ERROR.md: Comprehensive context for debugging
- Documents HuggingFace deployment errors with /api/v1/agents endpoint

ROUTING_FIX_ANALYSIS.md ADDED
@@ -0,0 +1,159 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # SAAP API Routing Fix - Root Cause Analysis
2
+
3
+ ## πŸ” Problem Identified
4
+
5
+ **Error Pattern in HuggingFace:**
6
+ ```
7
+ File "/app/spa_static_files.py", line 27, in get_response
8
+ raise RuntimeError("Not found")
9
+ RuntimeError: Not found
10
+ ```
11
+
12
+ ## βœ… Root Cause Discovery
13
+
14
+ **Deployed Code (HuggingFace - OLD):**
15
+ ```python
16
+ # Line 27 in deployed version
17
+ raise RuntimeError("Not found")
18
+ ```
19
+
20
+ **Local Code (Current - FIXED):**
21
+ ```python
22
+ try:
23
+ return await super().get_response(path, scope)
24
+ except Exception:
25
+ # File not found β†’ serve index.html for SPA routing
26
+ return FileResponse(os.path.join(self.directory, "index.html"))
27
+ ```
28
+
29
+ ## 🎯 Core Issue
30
+
31
+ **Version Mismatch:** HuggingFace deployment has old `spa_static_files.py` that raises exception instead of serving `index.html` for SPA fallback routing.
32
+
33
+ **Current Local Code Status:** βœ… ALREADY CORRECT
34
+
35
+ ## πŸ“‹ Routing Configuration Analysis
36
+
37
+ ### Current `backend/main.py` Structure (CORRECT):
38
+
39
+ ```python
40
+ # Line ~1200: All API routes defined first
41
+ @app.get("/api")
42
+ @app.get("/health")
43
+ @app.get("/api/v1/health")
44
+ @app.get("/api/v1/agents") # ← Critical agent list endpoint
45
+ @app.post("/api/v1/agents/{agent_id}/chat")
46
+ # ... all other API routes ...
47
+
48
+ # Line ~1890: Static files mounted LAST (correct order)
49
+ app.mount("/", SPAStaticFiles(directory=frontend_dist, html=True), name="static")
50
+ ```
51
+
52
+ ### Why This Works (Theory):
53
+
54
+ 1. **API Routes First**: FastAPI processes routes in definition order
55
+ 2. **Catch-All Last**: `app.mount("/", ...)` acts as catch-all for unmatched routes
56
+ 3. **SPAStaticFiles**: Serves static files OR index.html (SPA fallback)
57
+
58
+ ### Why It's Failing in HuggingFace:
59
+
60
+ **Old spa_static_files.py raises exception** β†’ FastAPI error handler β†’ 500 Internal Server Error
61
+
62
+ ## βœ… Solution
63
+
64
+ ### Option 1: Trigger Redeployment (RECOMMENDED)
65
+ ```bash
66
+ # Commit current (correct) code
67
+ git add backend/spa_static_files.py
68
+ git commit -m "fix(api): correct SPA fallback routing - serve index.html instead of RuntimeError"
69
+ git push huggingface main
70
+
71
+ # HuggingFace auto-deploys on push
72
+ ```
73
+
74
+ ### Option 2: Verify & Force Rebuild
75
+ ```bash
76
+ # Check deployed version
77
+ curl https://hwandji-saap.hf.space/api/v1/agents -v
78
+
79
+ # If still failing, force rebuild in HuggingFace UI:
80
+ # Settings β†’ Factory Reboot
81
+ ```
82
+
83
+ ## πŸ”§ Additional Safeguards
84
+
85
+ ### Add Explicit API Route Logging
86
+
87
+ **In `backend/main.py` after API route definitions:**
88
+ ```python
89
+ @app.on_event("startup")
90
+ async def log_routes():
91
+ """Log all registered routes for debugging"""
92
+ routes = []
93
+ for route in app.routes:
94
+ if hasattr(route, 'path'):
95
+ routes.append(f"{route.methods if hasattr(route, 'methods') else 'MOUNT'}: {route.path}")
96
+ logger.info(f"πŸ“ Registered Routes ({len(routes)}):")
97
+ for r in sorted(routes):
98
+ logger.info(f" {r}")
99
+ ```
100
+
101
+ ## πŸ§ͺ Testing Checklist
102
+
103
+ After deployment:
104
+
105
+ - [ ] `GET /api/v1/agents` returns 200 with agents list
106
+ - [ ] `GET /` returns Vue.js index.html (200)
107
+ - [ ] `GET /assets/*` returns static files (200/304)
108
+ - [ ] `GET /nonexistent` returns index.html (SPA routing)
109
+ - [ ] Frontend displays agents correctly
110
+
111
+ ## πŸ“Š Verification Commands
112
+
113
+ ```bash
114
+ # Test API endpoint
115
+ curl https://hwandji-saap.hf.space/api/v1/agents
116
+
117
+ # Expected response (200):
118
+ # {"agents": [...], "total": 7, "active": 7, ...}
119
+
120
+ # Test SPA fallback
121
+ curl https://hwandji-saap.hf.space/random-path -I
122
+
123
+ # Expected: 200 OK, Content-Type: text/html
124
+ ```
125
+
126
+ ## πŸš€ Deployment Steps
127
+
128
+ 1. **Commit Fix:**
129
+ ```bash
130
+ git status
131
+ git add backend/spa_static_files.py
132
+ git commit -m "fix(api): SPA fallback routing - FileResponse instead of RuntimeError"
133
+ ```
134
+
135
+ 2. **Push to HuggingFace:**
136
+ ```bash
137
+ git push huggingface main
138
+ ```
139
+
140
+ 3. **Monitor Deployment:**
141
+ - Watch HuggingFace build logs
142
+ - Wait for "Running" status
143
+ - Test `/api/v1/agents` endpoint
144
+
145
+ 4. **Verify Frontend:**
146
+ - Open https://hwandji-saap.hf.space
147
+ - Check agents display
148
+ - Test chat functionality
149
+
150
+ ## πŸ“ Summary
151
+
152
+ **Issue:** Old `spa_static_files.py` on HuggingFace raises `RuntimeError` β†’ 500 errors
153
+ **Local Status:** Code already fixed with `FileResponse` fallback βœ…
154
+ **Action Required:** Commit + Push to trigger HuggingFace redeployment
155
+ **Expected Resolution:** API routing works, agents display correctly
156
+
157
+ ---
158
+
159
+ **Next Step:** Execute deployment commands to push fixed code to HuggingFace
TASK_CONTEXT_API_ROUTING_ERROR.md ADDED
@@ -0,0 +1,244 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Task Context: HuggingFace API Routing Error
2
+
3
+ ## Current Work: Debugging Agent Display Failure on HuggingFace Deployment
4
+
5
+ ### Critical Error Summary
6
+
7
+ The SAAP platform deployed on HuggingFace is experiencing complete failure of agent display functionality. The frontend loads successfully, but agents cannot be retrieved from the backend.
8
+
9
+ **Error Pattern:**
10
+ ```
11
+ GET /api/v1/agents HTTP/1.1" 500 Internal Server Error
12
+ RuntimeError: Not found (from spa_static_files.py line 27)
13
+ ```
14
+
15
+ **Success Indicators:**
16
+ - Platform initializes successfully βœ…
17
+ - 7 agents loaded into database βœ…
18
+ - colossus connection successful βœ…
19
+ - OpenRouter client initialized βœ…
20
+ - Frontend static assets serve correctly βœ…
21
+ - API endpoint fails catastrophically ❌
22
+
23
+ ### Key Technical Concepts
24
+
25
+ **Architecture:**
26
+ - **Backend:** Python FastAPI with hybrid agent support (colossus + OpenRouter)
27
+ - **Frontend:** Vue.js SPA with Vite build system
28
+ - **Deployment:** Docker containerized on HuggingFace Spaces
29
+ - **Database:** SQLite with SQLAlchemy ORM
30
+ - **Agents:** 7 base templates (Jane, John, Lara, Theo, Justus, Leon, Luna Alesi)
31
+
32
+ **Critical Components:**
33
+ 1. `backend/main.py` - FastAPI application initialization and middleware ordering
34
+ 2. `backend/spa_static_files.py` - Custom static file handler (line 27 = error source)
35
+ 3. `backend/api/agents.py` - Agent API endpoints (should handle /api/v1/agents)
36
+ 4. `backend/services/agent_manager_hybrid.py` - Hybrid agent manager service
37
+
38
+ **Pydantic Warning (Secondary Issue):**
39
+ ```
40
+ UserWarning: Valid config keys have changed in V2:
41
+ * 'schema_extra' has been renamed to 'json_schema_extra'
42
+ ```
43
+
44
+ ### Relevant Files and Code
45
+
46
+ **Error Source Location:**
47
+ - File: `backend/spa_static_files.py`
48
+ - Line: 27
49
+ - Method: `get_response`
50
+ - Error: `raise RuntimeError("Not found")`
51
+
52
+ **Stack Trace Analysis:**
53
+ ```
54
+ Starlette Middleware Chain:
55
+ 1. proxy_headers.py
56
+ 2. CORS middleware
57
+ 3. Exception middleware
58
+ 4. Routing (fails here - goes to StaticFiles instead of API route)
59
+ 5. StaticFiles handler (spa_static_files.py line 105)
60
+ 6. get_response method (spa_static_files.py line 27) β†’ RuntimeError
61
+ ```
62
+
63
+ **Request Flow (ACTUAL):**
64
+ ```
65
+ Frontend β†’ GET /api/v1/agents
66
+ β†’ Starlette routing
67
+ β†’ StaticFiles middleware catches route (WRONG!)
68
+ β†’ spa_static_files.py:get_response()
69
+ β†’ RuntimeError("Not found")
70
+ ```
71
+
72
+ **Request Flow (EXPECTED):**
73
+ ```
74
+ Frontend β†’ GET /api/v1/agents
75
+ β†’ Starlette routing
76
+ β†’ FastAPI agent router
77
+ β†’ agent_api.py:get_agents()
78
+ β†’ Return agent list JSON
79
+ ```
80
+
81
+ ### Problem Solved Thus Far
82
+
83
+ **Initialization Success:**
84
+ - Database connection established
85
+ - 7 agents registered from templates
86
+ - colossus server connection verified (14.06s test successful)
87
+ - OpenRouter client initialized with free model fallback
88
+ - Hybrid agent manager operational
89
+
90
+ **Configuration Verified:**
91
+ - Environment: `OPENROUTER_API_KEY` warning present (expected in free mode)
92
+ - Database: `sqlite:///./saap_production.db` created successfully
93
+ - Providers: Both colossus (βœ…) and OpenRouter (βœ…) initialized
94
+
95
+ ### Pending Tasks and Next Steps
96
+
97
+ **Immediate Investigation Required:**
98
+
99
+ 1. **Examine spa_static_files.py implementation**
100
+ ```python
101
+ # Need to review line 27 in get_response method
102
+ # Understand why it's intercepting API routes
103
+ ```
104
+
105
+ 2. **Check main.py middleware ordering**
106
+ ```python
107
+ # Verify API router mounted BEFORE static file handler
108
+ # Pattern should be:
109
+ app.include_router(api_router) # FIRST
110
+ app.mount("/", SPAStaticFiles()) # LAST (catch-all)
111
+ ```
112
+
113
+ 3. **Verify route registration**
114
+ ```python
115
+ # Confirm /api/v1/agents endpoint properly registered
116
+ # Check FastAPI route debugging output
117
+ ```
118
+
119
+ 4. **Fix Pydantic V2 deprecation warning**
120
+ ```python
121
+ # Replace 'schema_extra' with 'json_schema_extra' in Pydantic models
122
+ ```
123
+
124
+ **Direct Quote from Error Logs:**
125
+
126
+ > "ich bekomme folgende Fehler in huggingface und die Aagenten werden aus diesem Grund nicht angezeigt:
127
+ >
128
+ > INFO: 10.20.38.235:38000 - "GET /api/v1/agents HTTP/1.1" 500 Internal Server Error
129
+ > ERROR: Exception in ASGI application
130
+ > Traceback (most recent call last):
131
+ > File "/app/spa_static_files.py", line 27, in get_response
132
+ > raise RuntimeError("Not found")
133
+ > RuntimeError: Not found"
134
+
135
+ **Where Development Left Off:**
136
+
137
+ The user encountered this error in production HuggingFace deployment. The platform starts successfully, all backend services initialize correctly, but the critical `/api/v1/agents` endpoint fails with a routing error. The static file handler is incorrectly intercepting API requests that should be handled by FastAPI routers.
138
+
139
+ **Root Cause Hypothesis:**
140
+
141
+ The middleware/route mounting order in `backend/main.py` likely has the SPA static file handler registered before (or with higher priority than) the API routes, causing all requests to be evaluated by the static file handler first. When `/api/v1/agents` doesn't match a static file, the handler raises `RuntimeError("Not found")` instead of allowing the request to continue to FastAPI routers.
142
+
143
+ **Required Fix Pattern:**
144
+
145
+ ```python
146
+ # backend/main.py - CORRECT PATTERN
147
+
148
+ app = FastAPI()
149
+
150
+ # 1. Register API routes FIRST (specific routes)
151
+ app.include_router(agent_router, prefix="/api/v1")
152
+ app.include_router(hybrid_router, prefix="/api/v1")
153
+
154
+ # 2. Mount static files LAST (catch-all fallback)
155
+ app.mount("/", SPAStaticFiles(directory="frontend/dist"))
156
+ ```
157
+
158
+ **Test Verification Needed:**
159
+ - Confirm `/api/v1/agents` returns JSON agent list (not 500)
160
+ - Verify static assets still load (/, /assets/*)
161
+ - Check agent display in frontend UI
162
+ - Validate both API and SPA work without conflicts
163
+
164
+ **Technical Context - File Structure:**
165
+ ```
166
+ backend/
167
+ β”œβ”€β”€ main.py (← FIX NEEDED: middleware ordering)
168
+ β”œβ”€β”€ spa_static_files.py (← line 27 raising error)
169
+ β”œβ”€β”€ api/
170
+ β”‚ β”œβ”€β”€ agents.py (← should handle /api/v1/agents)
171
+ β”‚ └── hybrid_endpoints.py
172
+ └── services/
173
+ └── agent_manager_hybrid.py
174
+ ```
175
+
176
+ **Environment Details:**
177
+ - Python: 3.11
178
+ - FastAPI: Latest (from requirements.txt)
179
+ - Uvicorn: Running on 0.0.0.0:7860
180
+ - Platform: HuggingFace Spaces
181
+ - Deployment: Docker container
182
+
183
+ **Error Frequency:**
184
+ Continuous - every frontend refresh attempts to fetch agents and fails. The error repeats every few seconds as the frontend retries the request.
185
+
186
+ ## Debugging Methodology to Apply
187
+
188
+ Per Rules/debugging-workflows.md - Four-Phase Framework:
189
+
190
+ **Phase 1: Root Cause Investigation** (IN PROGRESS)
191
+ - βœ… Read error messages - RuntimeError from spa_static_files.py line 27
192
+ - βœ… Stack trace analyzed - Starlette routing β†’ StaticFiles β†’ error
193
+ - ⚠️ Need to read spa_static_files.py and main.py to confirm hypothesis
194
+ - ⚠️ Need to trace middleware ordering
195
+
196
+ **Phase 2: Pattern Analysis** (PENDING)
197
+ - Find working FastAPI + SPA patterns
198
+ - Compare against current implementation
199
+ - Identify middleware ordering differences
200
+
201
+ **Phase 3: Hypothesis Testing** (PENDING)
202
+ - Hypothesis: Middleware ordering causes route interception
203
+ - Test: Reorder middleware in main.py
204
+ - Verify: API routes accessible, SPA still works
205
+
206
+ **Phase 4: Implementation** (PENDING)
207
+ - Create failing test for /api/v1/agents endpoint
208
+ - Fix middleware ordering
209
+ - Verify all tests pass
210
+ - Deploy and validate on HuggingFace
211
+
212
+ ## Integration Requirements
213
+
214
+ **Security Considerations:**
215
+ - API endpoints must not expose sensitive data
216
+ - CORS properly configured for HuggingFace domain
217
+ - Error messages sanitized (no stack traces to frontend)
218
+
219
+ **Code Quality Standards:**
220
+ - Fix Pydantic V2 deprecation warnings
221
+ - Follow FastAPI best practices for middleware ordering
222
+ - Ensure clear separation: API routes vs static file serving
223
+
224
+ **Testing Requirements:**
225
+ - Integration test: GET /api/v1/agents returns 200 + agent list
226
+ - Integration test: GET / returns 200 + SPA HTML
227
+ - Integration test: GET /assets/* returns 200 + static assets
228
+ - Coverage target: β‰₯75% (MCP/tool-heavy architecture acceptable)
229
+
230
+ ## Critical Files to Examine
231
+
232
+ 1. **backend/spa_static_files.py** - Line 27 error source
233
+ 2. **backend/main.py** - Middleware mounting order
234
+ 3. **backend/api/agents.py** - Agent endpoint implementation
235
+ 4. **backend/services/agent_manager_hybrid.py** - Agent retrieval logic
236
+
237
+ ## Success Criteria
238
+
239
+ - [ ] `/api/v1/agents` returns 200 status with valid JSON
240
+ - [ ] Agents display in frontend UI
241
+ - [ ] Static assets continue loading correctly
242
+ - [ ] No RuntimeError exceptions
243
+ - [ ] Pydantic deprecation warning resolved
244
+ - [ ] Route precedence clearly documented