392 lines
15 KiB
Markdown
392 lines
15 KiB
Markdown
# AGENTS.md — InfraSynth Base
|
|
|
|
## Project Overview
|
|
|
|
InfraSynth Base is a **reusable Django backend infrastructure kit** distributed as a single pip package (`infrasynth-base`). It provides 9 Django apps that cover authentication, authorization, audit logging, file storage, notifications, webhooks, workflows, job scheduling, feature flags, and billing. External systems (App B) install this package and build their domain apps on top without modifying InfraSynth source code.
|
|
|
|
**One version, one repo, one pip install.** Feature flags control what is active per tenant/user.
|
|
|
|
---
|
|
|
|
## Stack
|
|
|
|
| Component | Technology |
|
|
|-----------|-----------|
|
|
| Language | Python 3.12+ |
|
|
| Framework | Django 5.2+ |
|
|
| API | Django REST Framework 3.16+ |
|
|
| Database | PostgreSQL 16 |
|
|
| Cache/Broker | Redis |
|
|
| Task Queue | Celery 5.4+ (with django-celery-results + django-celery-beat) |
|
|
| Auth | JWT via HTTP-Only cookies (encrypted with Fernet) + API Keys |
|
|
| File Storage | S3, Cloudinary, GCS, local (via django-storages) |
|
|
| Payments | Stripe, MercadoPago, Wompi |
|
|
| 2FA | TOTP (pyotp + qrcode) |
|
|
| Anti-spam | ALTCHA (proof-of-work, self-hosted) |
|
|
| Monitoring | Flower (Celery dashboard) |
|
|
| Tests | pytest + pytest-django + factory-boy |
|
|
| Linting | ruff + mypy + pre-commit |
|
|
|
|
---
|
|
|
|
## Package Structure
|
|
|
|
```
|
|
infrasynth-base/
|
|
├── pyproject.toml # Root package metadata
|
|
├── docker-compose.yml
|
|
├── PLAN.md # Architecture blueprint
|
|
├── AGENTS.md # This file
|
|
│
|
|
├── infrasynth/ # Namespace package root
|
|
│ ├── shared/ # NOT a Django app. Zero-Django utilities.
|
|
│ ├── audit/ # Django app: 'infrasynth.audit'
|
|
│ ├── security/ # Django app: 'infrasynth.security'
|
|
│ ├── files/ # Django app: 'infrasynth.files'
|
|
│ ├── notifications/ # Django app: 'infrasynth.notifications'
|
|
│ ├── webhooks/ # Django app: 'infrasynth.webhooks'
|
|
│ ├── workflows/ # Django app: 'infrasynth.workflows'
|
|
│ ├── scheduler/ # Django app: 'infrasynth.scheduler'
|
|
│ ├── features/ # Django app: 'infrasynth.features'
|
|
│ └── billing/ # Django app: 'infrasynth.billing'
|
|
│
|
|
└── tests/
|
|
├── conftest.py
|
|
├── test_audit/
|
|
├── test_security/
|
|
├── test_files/
|
|
├── test_notifications/
|
|
├── test_webhooks/
|
|
├── test_workflows/
|
|
├── test_scheduler/
|
|
├── test_features/
|
|
└── test_billing/
|
|
```
|
|
|
|
---
|
|
|
|
## Architecture Principles
|
|
|
|
### 1. Zero Cross-App Import Rule
|
|
|
|
**No Django app that depends on another Django app may import from it directly.** The only allowed intra-app imports are:
|
|
- `infrasynth.shared.*` (protocols, enums, crypto, types)
|
|
- Django stdlib (`django.db.models`, `django.conf.settings`, `django.dispatch.Signal`)
|
|
|
|
### 2. Integration Mechanisms (in priority order)
|
|
|
|
| Mechanism | When to use | Example |
|
|
|-----------|------------|---------|
|
|
| **Settings dict** | Configure which concrete class/backend to use | `INFRASYNTH_NOTIFICATIONS["CHANNELS"]["email"]` points to SMTPChannel |
|
|
| **Signals** | Loose async communication between apps | `billing` emits `payment_succeeded`, App B's receiver sends email via `notifications` |
|
|
| **Registries** | Apps self-register capabilities at startup | `EventRegistry.register("helpdesk.ticket.created")` in `apps.py:ready()` |
|
|
| **ABCs/Protocols** | Define swappable interfaces | `BasePaymentGateway`, `BaseChannel`, `DataValidatorProtocol` |
|
|
| **ForeignKey (SET_NULL)** | Weak model coupling | `StoredFile` referenced by any model, on_delete=SET_NULL, related_name="+" |
|
|
| **AUTH_USER_MODEL** | Reference the user model | Always `settings.AUTH_USER_MODEL`, never `auth.User` directly |
|
|
| **FeatureService** | Cross-cutting enable/disable | `FeatureService().is_enabled("billing")` gates billing views |
|
|
|
|
### 3. Dependency Graph
|
|
|
|
```
|
|
infrasynth.shared ← Zero deps (protocols, enums, crypto)
|
|
↑
|
|
infrasynth.audit ← shared only
|
|
↑
|
|
All other Django apps ← shared + audit only
|
|
↑
|
|
infrasynth.features ← Used by ALL apps for feature gating
|
|
↑ (but apps register flags, don't import features)
|
|
```
|
|
|
|
### 4. Feature Flags Are the Orchestrator
|
|
|
|
`infrasynth.features` is the only app that is **always active**. Every other feature (module, endpoint, UI element) should be gated behind a feature flag. The frontend consumes `GET /api/features/active/` once at boot and renders conditionally.
|
|
|
|
Each app registers its flags in `apps.py:ready()`:
|
|
|
|
```python
|
|
class MyAppConfig(AppConfig):
|
|
def ready(self):
|
|
from infrasynth.features.registry import FeatureRegistry
|
|
FeatureRegistry.register("my_app.feature_x", default=True)
|
|
```
|
|
|
|
---
|
|
|
|
## Development Conventions
|
|
|
|
### Django App Structure
|
|
|
|
Every Django app follows this layout:
|
|
|
|
```
|
|
app_name/
|
|
├── __init__.py
|
|
├── apps.py # AppConfig: name, feature_flag, ready() for registry registrations
|
|
├── models.py # Django models
|
|
├── services.py # Business logic (pure Python, no DRF)
|
|
├── urls.py # URL patterns
|
|
├── serializers.py # DRF serializers
|
|
├── views.py # DRF views
|
|
├── filters.py # DRF filtersets
|
|
├── signals.py # Signal definitions (Signal() instances)
|
|
├── middleware.py # Django middleware (if needed)
|
|
├── tasks.py # Celery tasks (if needed)
|
|
└── migrations/ # Django migrations
|
|
```
|
|
|
|
### Model Conventions
|
|
|
|
1. **All models use `db_table` prefix:** `audit_model_change_log`, `security_api_key`, `files_stored_file`, etc.
|
|
2. **ForeignKey always uses `SET_NULL`** with `null=True, blank=True` unless cascade is semantically required.
|
|
3. **`related_name="+"`** on FK to other apps' models to avoid reverse relation clutter.
|
|
4. **`settings.AUTH_USER_MODEL`** for user references. Never hardcode `auth.User`.
|
|
5. **JSONField for flexible metadata**, not TextField.
|
|
6. **Use `infrasynth.shared.enums`** for choice fields (never hardcode strings in choices).
|
|
|
|
### Serializer Conventions
|
|
|
|
1. **FK fields need `{field}_info`** read-only serialized representations (for frontend display).
|
|
2. **Audit fields** (`created_by`, `created_at`, `updated_by`, `updated_at`) when present must be in `read_only_fields` and are populated by signals (not in `ModeloAuditable` base class since we avoid model inheritance).
|
|
3. **JSONField fields** need explicit serialization handling (the frontend expects objects, not strings).
|
|
4. **Use `SerializerMethodField`** sparingly — prefer annotations in the queryset.
|
|
|
|
### View Conventions
|
|
|
|
1. **All views are `ModelViewSet`** unless they have no model backing.
|
|
2. **Always set `permission_classes = [IsAuthenticated]`** plus specific permission classes.
|
|
3. **Always use `select_related()`/`prefetch_related()`** in `get_queryset()` to avoid N+1 queries.
|
|
4. **Feature flag check** in `initial()` method for gated views:
|
|
|
|
```python
|
|
def initial(self, request, *args, **kwargs):
|
|
if not FeatureService().is_enabled("billing", user=request.user):
|
|
raise NotFound()
|
|
super().initial(request, *args, **kwargs)
|
|
```
|
|
|
|
5. **Pagination:** All list views use the standard `CustomPagination` class. Query param `?page_size=` (default 25, max 100).
|
|
6. **Filtering:** Use `DjangoFilterBackend` with a `FilterSet` class per view.
|
|
|
|
### Signal Conventions
|
|
|
|
1. **Define signals in `signals.py`** as module-level `Signal()` instances.
|
|
2. **Receiver functions go in `receivers.py` or `apps.py:ready()`** (for connecting signals across apps).
|
|
3. **Always use `sender=` parameter** when connecting to specific model signals.
|
|
4. **Use `@receiver(signal_name)`** decorator pattern.
|
|
|
|
### Registry Conventions
|
|
|
|
Registries are singleton classes (not instances) with `@classmethod` methods. They live in a `registry.py` file in their owning app:
|
|
|
|
- `infrasynth.features.registry.FeatureRegistry` — feature flag definitions
|
|
- `infrasynth.webhooks.registry.EventRegistry` — event definitions
|
|
- `infrasynth.notifications.resolvers.VariableResolverRegistry` — template variable resolvers
|
|
- `infrasynth.workflows.validators.DataValidatorRegistry` — workflow data validators
|
|
|
|
Pattern:
|
|
```python
|
|
class MyRegistry:
|
|
_items: dict = {}
|
|
|
|
@classmethod
|
|
def register(cls, key, **kwargs):
|
|
cls._items[key] = kwargs
|
|
|
|
@classmethod
|
|
def get(cls, key):
|
|
return cls._items.get(key)
|
|
|
|
@classmethod
|
|
def get_all(cls):
|
|
return dict(cls._items)
|
|
```
|
|
|
|
### Testing Conventions
|
|
|
|
1. **Use pytest** with `pytest-django` (`pytest.mark.django_db`).
|
|
2. **Use factory-boy** for model factories (`tests/factories.py` in each app test directory).
|
|
3. **API tests use `APIClient`** from DRF with JWT cookies set manually.
|
|
4. **Test structure:**
|
|
- `test_models.py` — model creation, validation, constraints
|
|
- `test_services.py` — business logic
|
|
- `test_views.py` — API endpoints (auth, permissions, CRUD, edge cases)
|
|
- `test_signals.py` — signal emission and receiver behavior
|
|
- `test_integration.py` — cross-app communication (registries, signals)
|
|
|
|
5. **Conftest fixtures:**
|
|
- `api_client` — DRF APIClient
|
|
- `authenticated_client` — APIClient with JWT cookies set
|
|
- `admin_client` — authenticated superuser client
|
|
- `user_factory`, `role_factory`, etc.
|
|
|
|
### Settings Conventions
|
|
|
|
1. **All InfraSynth settings use the prefix `INFRASYNTH_`** followed by the app name in uppercase.
|
|
2. **Settings are dicts**, not flat keys: `INFRASYNTH_SECURITY = {"COOKIE_SECURE": True}`.
|
|
3. **Every setting has a sensible default** — the system must run with zero configuration in development.
|
|
4. **Read settings with the helper** (not `getattr` directly):
|
|
|
|
```python
|
|
from infrasynth.shared.settings_utils import get_setting
|
|
cookie_secure = get_setting("INFRASYNTH_SECURITY", "COOKIE_SECURE", True)
|
|
```
|
|
|
|
### Crypto Conventions
|
|
|
|
1. **Use `infrasynth.shared.crypto`** for Fernet encryption/decryption.
|
|
2. **Encrypt secrets at rest:** API keys, SMTP passwords, payment gateway credentials.
|
|
3. **Never log encrypted values** — log the fact of encryption, not the ciphertext or plaintext.
|
|
4. **CRYPTO_KEY** must be set in environment. Auto-generate in dev if missing (warn loudly).
|
|
|
|
### Migration Conventions
|
|
|
|
1. **Apps are namespaced** in migrations to avoid collisions:
|
|
- `infrasynth.audit.migrations`
|
|
- `infrasynth.security.migrations`
|
|
2. **Never use `RunPython`** with model imports — use `apps.get_model()`.
|
|
3. **Data migrations** go in separate migration files from schema migrations.
|
|
|
|
---
|
|
|
|
## Integration Examples for External App B
|
|
|
|
### App B needs: custom notification channel
|
|
|
|
```python
|
|
# helpdesk/channels.py
|
|
from infrasynth.notifications.channels.base import BaseChannel
|
|
|
|
class SlackChannel(BaseChannel):
|
|
channel_type = "slack"
|
|
|
|
def send(self, recipient, subject, body, is_html=True, attachments=None):
|
|
# Send to Slack webhook
|
|
...
|
|
return Result.ok(True)
|
|
|
|
# settings.py
|
|
INFRASYNTH_NOTIFICATIONS = {
|
|
"CHANNELS": {
|
|
"slack": {
|
|
"primary": "helpdesk.channels.SlackChannel",
|
|
},
|
|
},
|
|
}
|
|
```
|
|
|
|
### App B needs: webhook handler for a new external service
|
|
|
|
```python
|
|
# helpdesk/webhook_handlers.py
|
|
from infrasynth.webhooks.inbound.handlers import BaseInboundHandler
|
|
|
|
class JiraWebhookHandler(BaseInboundHandler):
|
|
def verify(self, payload, headers, secret):
|
|
# Verify Jira HMAC
|
|
...
|
|
|
|
def process(self, event_type, payload):
|
|
# Sync Jira issue to local Ticket model
|
|
...
|
|
|
|
# Register via admin or data migration: InboundEndpoint(slug="jira", handler="helpdesk.webhook_handlers.JiraWebhookHandler")
|
|
```
|
|
|
|
### App B needs: workflow data validation for its domain
|
|
|
|
```python
|
|
# helpdesk/validators.py
|
|
class TicketDataValidator:
|
|
def validate(self, node, data, context):
|
|
if not data.get("resolution_note"):
|
|
raise ValidationError({"resolution_note": "Required when resolving."})
|
|
return data
|
|
|
|
# helpdesk/apps.py → ready():
|
|
DataValidatorRegistry.register("ticket_approval", TicketDataValidator())
|
|
```
|
|
|
|
---
|
|
|
|
## Common Patterns and Anti-Patterns
|
|
|
|
### ✅ DO
|
|
|
|
- Use `settings.AUTH_USER_MODEL` for all user references
|
|
- Use signals for cross-app communication
|
|
- Register events/resolvers/validators in `apps.py:ready()`
|
|
- Gate views/endpoints behind feature flags
|
|
- Use `on_delete=SET_NULL` with `null=True, blank=True` for cross-app FKs
|
|
- Use `related_name="+"` for FKs to models in other apps
|
|
- Use `db_table` prefix for all models
|
|
- Encrypt secrets at rest with Fernet
|
|
- Use `Result[T, E]` monad for service methods that can fail
|
|
- Add `select_related()`/`prefetch_related()` in every view's `get_queryset()`
|
|
|
|
### ❌ DON'T
|
|
|
|
- Don't import models from one Django app into another Django app
|
|
- Don't hardcode `auth.User` — use `settings.AUTH_USER_MODEL`
|
|
- Don't use `on_delete=CASCADE` on cross-app FKs
|
|
- Don't bypass the FeatureRegistry — always register flags
|
|
- Don't hardcode channel URLs, gateway credentials, or SMTP settings in code
|
|
- Don't log plaintext secrets, tokens, or passwords
|
|
- Don't use signals for synchronous request-response flows (use direct method calls)
|
|
- Don't create circular imports — if app A needs app B, and app B needs app A, refactor into shared or use signals
|
|
- Don't store file contents in the database — always use the files app's storage abstraction
|
|
|
|
---
|
|
|
|
## Key Files Reference
|
|
|
|
| File | Purpose |
|
|
|------|---------|
|
|
| `infrasynth/shared/protocols.py` | All ABCs and Protocols |
|
|
| `infrasynth/shared/crypto.py` | Fernet encrypt/decrypt/rotation |
|
|
| `infrasynth/shared/enums.py` | All shared enums |
|
|
| `infrasynth/shared/results.py` | Result monad |
|
|
| `infrasynth/features/registry.py` | Feature flag registry |
|
|
| `infrasynth/features/services.py` | Feature flag evaluation |
|
|
| `infrasynth/webhooks/registry.py` | Event registry |
|
|
| `infrasynth/notifications/resolvers.py` | Template variable resolvers |
|
|
| `infrasynth/notifications/channels/base.py` | Channel ABC |
|
|
| `infrasynth/billing/gateways/base.py` | Payment gateway ABC |
|
|
| `infrasynth/workflows/validators.py` | Data validator protocol + registry |
|
|
| `infrasynth/workflows/models.py` | WorkflowAwareModel abstract mixin |
|
|
| `infrasynth/security/services.py` | AuthorizationService |
|
|
| `infrasynth/security/auth/cookies.py` | CookieJWTAuthentication |
|
|
| `infrasynth/security/auth/api_keys.py` | APIKeyAuthentication |
|
|
| `infrasynth/security/permissions.py` | HybridPermission + require_permission |
|
|
| `infrasynth/files/services.py` | FileService (upload, signed_url, delete) |
|
|
| `infrasynth/scheduler/services.py` | TaskService |
|
|
|
|
---
|
|
|
|
## Setup for Development
|
|
|
|
```bash
|
|
# Clone
|
|
git clone <repo-url> && cd infrasynth-base
|
|
|
|
# Virtual environment
|
|
python -m venv .venv && source .venv/bin/activate
|
|
|
|
# Install with dev dependencies
|
|
pip install -e ".[dev]"
|
|
|
|
# Start services
|
|
docker compose up -d db redis
|
|
|
|
# Run migrations
|
|
python manage.py migrate
|
|
|
|
# Run tests
|
|
pytest
|
|
|
|
# Run linter
|
|
ruff check .
|
|
|
|
# Run type checker
|
|
mypy infrasynth/
|
|
```
|