# AGENTS.md — InfraSynth Base

## Project Overview

InfraSynth Base is a **reusable Django backend infrastructure kit** distributed as a single pip package (`infrasynth-base`). It is the **one shared kit** every Infrasynth app depends on, providing 11 Django apps that cover tenancy, authentication, authorization, audit logging, file storage, notifications, webhooks, workflows, job scheduling, feature flags, typed tenant configuration, and billing. External systems (App B) install this package and build their domain apps on top without modifying InfraSynth source code.

**Every app is multi-tenant.** One deployment per app serves all customers; a customer is a **tenant** (workspace), isolated at row level via `tenant_id` on a shared schema. Read `../TENANCY.md` — it is the source of truth for tenancy.

**No license server.** Everything runs on our own infrastructure, so there are no signed license keys, no phone-home, no offline SDK, and no validation grace period. What a tenant may use is an **entitlement**, enforced in-process by `infrasynth.billing`. Read `../ENTITLEMENTS.md`.

**One version, one repo, one pip install.** Feature flags are operational toggles per tenant/user; entitlements are commercial rights.

---

## Stack

| Component | Technology |
|-----------|-----------|
| Language | Python 3.12+ |
| Framework | Django 5.2+ |
| API | Django REST Framework 3.16+ |
| Database | PostgreSQL 16 |
| Cache/Broker | Redis |
| Task Queue | Celery 5.4+ (with django-celery-results + django-celery-beat) |
| Auth | JWT via HTTP-Only cookies (encrypted with Fernet) + API Keys |
| File Storage | S3, Cloudinary, GCS, local (via django-storages) |
| Payments | Stripe, MercadoPago, Wompi |
| 2FA | TOTP (pyotp + qrcode) |
| Anti-spam | ALTCHA (proof-of-work, self-hosted) |
| Monitoring | Flower (Celery dashboard) |
| Tests | pytest + pytest-django + factory-boy |
| Linting | ruff + mypy + pre-commit |

---

## Package Structure

```
infrasynth-base/
├── pyproject.toml                 # Root package metadata
├── docker-compose.yml
├── PLAN.md                        # Architecture blueprint
├── AGENTS.md                      # This file
│
├── infrasynth/                    # Namespace package root
│   ├── shared/                    # NOT a Django app. Zero-Django utilities.
│   ├── api/                       # DRF API layer (envelope, camelCase, cursor pagination, request-id)
│   ├── tenancy/                   # Django app: 'infrasynth.tenancy' (Tenant, membership, scoping)
│   ├── audit/                     # Django app: 'infrasynth.audit'
│   ├── security/                  # Django app: 'infrasynth.security'
│   ├── files/                     # Django app: 'infrasynth.files'
│   ├── notifications/             # Django app: 'infrasynth.notifications'
│   ├── webhooks/                  # Django app: 'infrasynth.webhooks'
│   ├── workflows/                 # Django app: 'infrasynth.workflows'
│   ├── scheduler/                 # Django app: 'infrasynth.scheduler'
│   ├── features/                  # Django app: 'infrasynth.features'
│   ├── configs/                   # Django app: 'infrasynth.configs' (typed per-tenant configuration)
│   └── billing/                   # Django app: 'infrasynth.billing'
│
└── tests/
    ├── conftest.py
    ├── test_audit/
    ├── test_security/
    ├── test_files/
    ├── test_notifications/
    ├── test_webhooks/
    ├── test_workflows/
    ├── test_scheduler/
    ├── test_features/
    └── test_billing/
```

---

## Architecture Principles

### 1. Zero Cross-App Import Rule

**No Django app that depends on another Django app may import from it directly.** The only allowed intra-app imports are:
- `infrasynth.shared.*` (protocols, enums, crypto, types)
- Django stdlib (`django.db.models`, `django.conf.settings`, `django.dispatch.Signal`)

### 1b. Multi-Tenancy First — read `../TENANCY.md` before any model

Every app is multi-tenant. The kit's `infrasynth.tenancy` app provides the tenant model, membership, request context, and scoped managers. Non-negotiables:

- Every **tenant-owned** model has a non-null `tenant` FK, `objects = TenantManager()` and `all_objects = AllObjectsManager()`.
- No tenant context ⇒ the scoped manager returns an **empty queryset** (fail closed). A query that works without a tenant is a bug.
- Cross-tenant object access returns **`404`, never `403`**.
- Uniqueness that was global becomes unique **per tenant**; indexes lead with `tenant_id`.
- `unsafe_all()` is never called from a view.
- Celery tasks and signals carry `tenant_id` explicitly; cache keys are prefixed `tenant:{id}:`.

See `../TENANCY.md` §4 for the full contract and the per-model scoping table in `PLAN.md` §2.0.

### 2. Integration Mechanisms (in priority order)

| Mechanism | When to use | Example |
|-----------|------------|---------|
| **Settings dict** | Configure which concrete class/backend to use | `INFRASYNTH_NOTIFICATIONS["CHANNELS"]["email"]` points to SMTPChannel |
| **Signals** | Loose async communication between apps | `billing` emits `payment_succeeded`, App B's receiver sends email via `notifications` |
| **Registries** | Apps self-register capabilities at startup | `EventRegistry.register("helpdesk.ticket.created")` in `apps.py:ready()` |
| **ABCs/Protocols** | Define swappable interfaces | `BasePaymentGateway`, `BaseChannel`, `DataValidatorProtocol` |
| **ForeignKey (SET_NULL)** | Weak model coupling | `StoredFile` referenced by any model, on_delete=SET_NULL, related_name="+" |
| **AUTH_USER_MODEL** | Reference the user model | Always `settings.AUTH_USER_MODEL`, never `auth.User` directly |
| **FeatureService** | Cross-cutting enable/disable | `FeatureService().is_enabled("billing")` gates billing views |

### 3. Dependency Graph

```
infrasynth.shared          ← Zero deps (protocols, enums, crypto)
    ↑
infrasynth.audit           ← shared only
    ↑
infrasynth.tenancy         ← shared + audit (defines isolation; used by every tenant-owned app)
    ↑
All other Django apps      ← shared + audit (+ tenancy where tenant-owned)
    ↑
infrasynth.features        ← Used by ALL apps for feature gating
    ↑                       (but apps register flags, don't import features)
```

### 4. Feature Flags Are the Orchestrator

`infrasynth.features` is the only app that is **always active**. Every other feature (module, endpoint, UI element) should be gated behind a feature flag. The frontend consumes `GET /api/features/active/` once at boot and renders conditionally.

Each app registers its flags in `apps.py:ready()`:

```python
class MyAppConfig(AppConfig):
    def ready(self):
        from infrasynth.features.registry import FeatureRegistry
        FeatureRegistry.register("my_app.feature_x", default=True)
```

---

## Development Conventions

### Django App Structure

Every Django app follows this layout:

```
app_name/
├── __init__.py
├── apps.py               # AppConfig: name, feature_flag, ready() for registry registrations
├── models.py             # Django models
├── services.py           # Business logic (pure Python, no DRF)
├── urls.py               # URL patterns
├── serializers.py        # DRF serializers
├── views.py              # DRF views
├── filters.py            # DRF filtersets
├── signals.py            # Signal definitions (Signal() instances)
├── middleware.py          # Django middleware (if needed)
├── tasks.py              # Celery tasks (if needed)
└── migrations/           # Django migrations
```

### Model Conventions

1. **All models use `db_table` prefix:** `audit_model_change_log`, `security_api_key`, `files_stored_file`, etc.
2. **ForeignKey always uses `SET_NULL`** with `null=True, blank=True` unless cascade is semantically required.
3. **`related_name="+"`** on FK to other apps' models to avoid reverse relation clutter.
4. **`settings.AUTH_USER_MODEL`** for user references. Never hardcode `auth.User`.
5. **JSONField for flexible metadata**, not TextField.
6. **Use `infrasynth.shared.enums`** for choice fields (never hardcode strings in choices).
7. **Tenant-owned models carry `tenant` + scoped managers:** non-null FK to `tenancy.Tenant`, `objects = TenantManager()`, `all_objects = AllObjectsManager()`. Uniqueness becomes `(tenant, field)` and indexes lead with `tenant_id` (`../TENANCY.md` §4).
8. **Prefer the tenancy mixins over hand-writing the field:** inherit `infrasynth.tenancy.mixins.TenantOwnedModel` (non-null `tenant`, `TenantManager` default, `all_objects`, and save-time tenant auto-assignment) or `GlobalOrTenantModel` (nullable `tenant`, `GlobalOrTenantManager` returning global + current-tenant rows, `resolve()` for precedence). Do not redeclare `tenant`/managers on a model that already inherits a mixin.

### Serializer Conventions

1. **FK fields need `{field}_info`** read-only serialized representations (for frontend display).
2. **Audit fields** (`created_by`, `created_at`, `updated_by`, `updated_at`) when present must be in `read_only_fields` and are populated by signals (not in `ModeloAuditable` base class since we avoid model inheritance).
3. **JSONField fields** need explicit serialization handling (the frontend expects objects, not strings).
4. **Use `SerializerMethodField`** sparingly — prefer annotations in the queryset.

### View Conventions

1. **All views are `ModelViewSet`** unless they have no model backing.
2. **Always set `permission_classes = [IsAuthenticated]`** plus specific permission classes.
3. **Always use `select_related()`/`prefetch_related()`** in `get_queryset()` to avoid N+1 queries.
4. **Feature flag check** in `initial()` method for gated views:

```python
def initial(self, request, *args, **kwargs):
    if not FeatureService().is_enabled("billing", user=request.user):
        raise NotFound()
    super().initial(request, *args, **kwargs)
```

5. **Pagination:** All list views use the standard `CustomPagination` class. Query param `?page_size=` (default 25, max 100).
6. **Filtering:** Use `DjangoFilterBackend` with a `FilterSet` class per view.
7. **Tenant scoping is automatic:** never filter by tenant by hand — `Model.objects` is already scoped to the current tenant. Never call `unsafe_all()` from a view. A cross-tenant id resolves to `404` (the scoped manager makes the row invisible), never `403`.
8. **Permissions are automatic:** subclass `infrasynth.security.viewsets.InfraSynthModelViewSet` (or `InfraSynthReadOnlyModelViewSet`) and the derived `{app}.{verb}_{model}` codename is enforced per action — no `required_permissions` needed. Override with an explicit `required_permissions` (any-of; set `require_all = True` for all-of) or map a custom action with `action_permissions = {"action": "codename"}`. Register non-model permissions with `PermissionRegistry.register(...)` in `apps.py:ready()`.

### Signal Conventions

1. **Define signals in `signals.py`** as module-level `Signal()` instances.
2. **Receiver functions go in `receivers.py` or `apps.py:ready()`** (for connecting signals across apps).
3. **Always use `sender=` parameter** when connecting to specific model signals.
4. **Use `@receiver(signal_name)`** decorator pattern.
5. **Every declared `Signal()` is emitted.** The kit's signals all have a real
   `.send(...)` call site (`features.*`, `scheduler.task_completed/task_failed`,
   `tenancy.*`, `audit.model_changed`, `configs.config_changed/config_reset`).
   Emit from the service/mutation point, not from a receiver, and always pass
   `tenant_id` explicitly. Never put a secret's plaintext or ciphertext in a
   signal payload — pass it masked (`None`).

### Registry Conventions

Registries are singleton classes (not instances) with `@classmethod` methods. They live in a `registry.py` file in their owning app:

- `infrasynth.features.registry.FeatureRegistry` — feature flag definitions
- `infrasynth.security.registry.PermissionRegistry` — custom permission definitions
- `infrasynth.configs.registry.ConfigRegistry` — typed configuration definitions
- `infrasynth.webhooks.registry.EventRegistry` — event definitions
- `infrasynth.notifications.resolvers.VariableResolverRegistry` — template variable resolvers
- `infrasynth.workflows.validators.DataValidatorRegistry` — workflow data validators

Pattern:
```python
class MyRegistry:
    _items: dict = {}

    @classmethod
    def register(cls, key, **kwargs):
        cls._items[key] = kwargs

    @classmethod
    def get(cls, key):
        return cls._items.get(key)

    @classmethod
    def get_all(cls):
        return dict(cls._items)
```

### Testing Conventions

1. **Use pytest** with `pytest-django` (`pytest.mark.django_db`).
2. **Use factory-boy** for model factories (`tests/factories.py` in each app test directory).
3. **API tests use `APIClient`** from DRF with JWT cookies set manually.
4. **Test structure:**
   - `test_models.py` — model creation, validation, constraints
   - `test_services.py` — business logic
   - `test_views.py` — API endpoints (auth, permissions, CRUD, edge cases)
   - `test_signals.py` — signal emission and receiver behavior
   - `test_integration.py` — cross-app communication (registries, signals)

5. **Conftest fixtures:**
   - `api_client` — DRF APIClient
   - `authenticated_client` — APIClient with JWT cookies set
   - `admin_client` — authenticated superuser client
   - `user_factory`, `role_factory`, etc.
   - `tenant_factory`, `membership_factory` — for multi-tenant tests

6. **Tenant isolation is mandatory:** create two tenants with data and assert tenant A cannot read, write, update, or delete tenant B's rows, and that cross-tenant access returns `404`. Any Celery task touching tenant data gets a test proving it carries `tenant_id` and does not leak across tenants.

### Settings Conventions

1. **All InfraSynth settings use the prefix `INFRASYNTH_`** followed by the app name in uppercase.
2. **Settings are dicts**, not flat keys: `INFRASYNTH_SECURITY = {"COOKIE_SECURE": True}`.
3. **Every setting has a sensible default** — the system must run with zero configuration in development.
4. **Read settings with the helper** (not `getattr` directly):

```python
from infrasynth.shared.settings_utils import get_setting
cookie_secure = get_setting("INFRASYNTH_SECURITY", "COOKIE_SECURE", True)
```

5. **Tenancy is configured via `INFRASYNTH_TENANCY`** (`TENANT_MODEL`, `TENANT_CLAIM`, `REQUIRE_TENANT_BY_DEFAULT`, allowlist, defaults). Billing/grace via `INFRASYNTH_BILLING` (`GRACE_PERIOD_DAYS`, `DEFAULT_CURRENCY`). Typed tenant configuration via `INFRASYNTH_CONFIGS` (`DEFINITIONS`, cache, `ALLOW_GLOBAL_WRITES`). Permissions via `INFRASYNTH_SECURITY` (`AUTO_PERMISSIONS`, `STRICT_PERMISSION_VALIDATION`, `PERMISSION_EXCLUDE_MODELS`). All have safe defaults (`../TENANCY.md`, `../ENTITLEMENTS.md`).

### Crypto Conventions

1. **Use `infrasynth.shared.crypto`** for Fernet encryption/decryption.
2. **Encrypt secrets at rest:** API keys, SMTP passwords, payment gateway credentials.
3. **Never log encrypted values** — log the fact of encryption, not the ciphertext or plaintext.
4. **CRYPTO_KEY** must be set in environment. Auto-generate in dev if missing (warn loudly).

### Migration Conventions

1. **Apps are namespaced** in migrations to avoid collisions:
   - `infrasynth.audit.migrations`
   - `infrasynth.security.migrations`
2. **Never use `RunPython`** with model imports — use `apps.get_model()`.
3. **Data migrations** go in separate migration files from schema migrations.

---

## Integration Examples for External App B

### App B needs: custom notification channel

```python
# helpdesk/channels.py
from infrasynth.notifications.channels.base import BaseChannel

class SlackChannel(BaseChannel):
    channel_type = "slack"

    def send(self, recipient, subject, body, is_html=True, attachments=None):
        # Send to Slack webhook
        ...
        return Result.ok(True)

# settings.py
INFRASYNTH_NOTIFICATIONS = {
    "CHANNELS": {
        "slack": {
            "primary": "helpdesk.channels.SlackChannel",
        },
    },
}
```

### App B needs: webhook handler for a new external service

```python
# helpdesk/webhook_handlers.py
from infrasynth.webhooks.inbound.handlers import BaseInboundHandler

class JiraWebhookHandler(BaseInboundHandler):
    def verify(self, payload, headers, secret):
        # Verify Jira HMAC
        ...

    def process(self, event_type, payload):
        # Sync Jira issue to local Ticket model
        ...

# Register via admin or data migration: InboundEndpoint(slug="jira", handler="helpdesk.webhook_handlers.JiraWebhookHandler")
```

### App B needs: workflow data validation for its domain

```python
# helpdesk/validators.py
class TicketDataValidator:
    def validate(self, node, data, context):
        if not data.get("resolution_note"):
            raise ValidationError({"resolution_note": "Required when resolving."})
        return data

# helpdesk/apps.py → ready():
DataValidatorRegistry.register("ticket_approval", TicketDataValidator())
```

### App B needs: a tenant-owned model

```python
# helpdesk/models.py
from infrasynth.tenancy.managers import TenantManager, AllObjectsManager

class Ticket(models.Model):
    tenant = models.ForeignKey("tenancy.Tenant", on_delete=models.CASCADE, related_name="+")
    subject = models.CharField(max_length=255)

    objects = TenantManager()          # always scoped to the current tenant
    all_objects = AllObjectsManager()  # unscoped — admin/management only

    class Meta:
        constraints = [models.UniqueConstraint(fields=["tenant", "slug"], name="uniq_ticket_slug_per_tenant")]
```

No view filters by tenant by hand: `Ticket.objects.all()` already returns only the current tenant's tickets, and a foreign tenant's id yields `404`. Never call `unsafe_all()` in a view.

### App B needs: entitlement gating

```python
# helpdesk/views.py
from infrasynth.billing.services import EntitlementService
from infrasynth.shared.exceptions import EntitlementError

def create_ticket(request, tenant):
    if not EntitlementService().is_entitled(tenant, "helpdesk", feature="tickets"):
        raise EntitlementError(code="ENTITLEMENT_PLAN_UPGRADE_REQUIRED", app="helpdesk", feature="tickets")
    ...
```

Enforcement is server-side and in-process. There is no license key and no offline check (`../ENTITLEMENTS.md`).

---

## Common Patterns and Anti-Patterns

### ✅ DO

- Use `settings.AUTH_USER_MODEL` for all user references
- Use signals for cross-app communication
- Register events/resolvers/validators in `apps.py:ready()`
- Gate views/endpoints behind feature flags
- Use `on_delete=SET_NULL` with `null=True, blank=True` for cross-app FKs
- Use `related_name="+"` for FKs to models in other apps
- Use `db_table` prefix for all models
- Encrypt secrets at rest with Fernet
- Use `Result[T, E]` monad for service methods that can fail
- Add `select_related()`/`prefetch_related()` in every view's `get_queryset()`
- Put `tenant` + `TenantManager` on every tenant-owned model
- Resolve the tenant from the session token or a tenant-scoped credential
- Prefix every cache/Redis/rate-limit key with `tenant:{id}:`
- Carry `tenant_id` explicitly into Celery tasks and signals

### ❌ DON'T

- Don't import models from one Django app into another Django app
- Don't hardcode `auth.User` — use `settings.AUTH_USER_MODEL`
- Don't use `on_delete=CASCADE` on cross-app FKs
- Don't bypass the FeatureRegistry — always register flags
- Don't hardcode channel URLs, gateway credentials, or SMTP settings in code
- Don't log plaintext secrets, tokens, or passwords
- Don't use signals for synchronous request-response flows (use direct method calls)
- Don't create circular imports — if app A needs app B, and app B needs app A, refactor into shared or use signals
- Don't store file contents in the database — always use the files app's storage abstraction
- Don't resolve a tenant from a client-supplied id on an unauthenticated request
- Don't call `unsafe_all()` from a view
- Don't leave a formerly-global unique field global when it should be unique per tenant
- Don't introduce license keys, a license server, phone-home, or offline verification — use entitlements

---

## Key Files Reference

| File | Purpose |
|------|---------|
| `infrasynth/shared/protocols.py` | All ABCs and Protocols (incl. `TenantProtocol`) |
| `infrasynth/api/renderers.py` | EnvelopeJSONRenderer (envelope + camelCase) |
| `infrasynth/api/pagination.py` | CursorPagination |
| `infrasynth/api/exceptions.py` | envelope_exception_handler + namespaced error codes |
| `infrasynth/api/middleware.py` | RequestIdMiddleware |
| `infrasynth/tenancy/models.py` | Tenant, TenantMembership |
| `infrasynth/tenancy/managers.py` | TenantManager, AllObjectsManager |
| `infrasynth/tenancy/middleware.py` | TenantMiddleware (resolves the tenant from the token claim) |
| `infrasynth/tenancy/context.py` | `current_tenant` ContextVar |
| `infrasynth/billing/models.py` | App, Plan, Entitlement, Subscription, Invoice, PaymentTransaction |
| `infrasynth/shared/crypto.py` | Fernet encrypt/decrypt/rotation |
| `infrasynth/shared/enums.py` | All shared enums |
| `infrasynth/shared/results.py` | Result monad |
| `infrasynth/features/registry.py` | Feature flag registry |
| `infrasynth/features/services.py` | Feature flag evaluation |
| `infrasynth/configs/registry.py` | Typed configuration registry |
| `infrasynth/configs/services.py` | ConfigService (precedence, secrets, cache) |
| `infrasynth/security/registry.py` | Custom permission registry |
| `infrasynth/security/catalog.py` | Permission derivation + `sync_permissions` |
| `infrasynth/security/permissions.py` | HybridPermission + AutoPermission |
| `infrasynth/security/viewsets.py` | Auto-permission viewset bases |
| `infrasynth/webhooks/registry.py` | Event registry |
| `infrasynth/notifications/resolvers.py` | Template variable resolvers |
| `infrasynth/notifications/channels/base.py` | Channel ABC |
| `infrasynth/billing/gateways/base.py` | Payment gateway ABC |
| `infrasynth/workflows/validators.py` | Data validator protocol + registry |
| `infrasynth/workflows/models.py` | WorkflowAwareModel abstract mixin |
| `infrasynth/security/services.py` | AuthorizationService |
| `infrasynth/security/auth/cookies.py` | CookieJWTAuthentication |
| `infrasynth/security/auth/api_keys.py` | APIKeyAuthentication |
| `infrasynth/security/permissions.py` | HybridPermission + AutoPermission |
| `infrasynth/files/services.py` | FileService (upload, signed_url, delete) |
| `infrasynth/scheduler/services.py` | TaskService |

---

## Setup for Development

```bash
# Clone
git clone <repo-url> && cd infrasynth-base

# Virtual environment
python -m venv .venv && source .venv/bin/activate

# Install with dev dependencies
pip install -e ".[dev]"

# Start services
docker compose up -d db redis

# Run migrations
python manage.py migrate

# Run tests
pytest

# Run linter
ruff check .

# Run type checker
mypy infrasynth/
```