infrasynth-backend-kit/README.md

# InfraSynth Base

Reusable, multi-tenant Django infrastructure kit — the one shared package every InfraSynth app depends on.

```bash
pip install -e ".[dev]"          # install the kit + dev tooling
docker compose up -d db redis    # postgres + redis
python manage.py migrate         # create the schema
python manage.py runserver       # http://localhost:8000/api/v1/schema/docs/
```

Then `pytest` (single-command test suite), `ruff check .`, and `mypy infrasynth/`.

---

## What this is

One pip package (`infrasynth-base`) providing eleven Django apps so no app ever reimplements auth, tenancy, entitlements, audit, files, notifications, webhooks, workflows, scheduling, configuration, or the API envelope:

| Module | Responsibility |
|---|---|
| `infrasynth.shared` | Zero-Django primitives: protocols, enums, `Result`, Fernet crypto, settings helper |
| `infrasynth.api` | DRF envelope, camelCase, cursor pagination, request-id, exceptions, throttling, idempotency, webhook hardening |
| `infrasynth.tenancy` | `Tenant`, membership, invitations, platform staff, `current_tenant`, scoped managers, middleware |
| `infrasynth.security` | JWT cookie + API-key auth, roles/grants/revokes, permission catalog, automatic model permissions, 2FA (TOTP), ALTCHA, login brute-force guard, password policy |
| `infrasynth.audit` | Passive create/update/delete tracking, API interaction log, security events, retention purge |
| `infrasynth.features` | Operational feature flags with tenant/user/group overrides, rollout %, environment targeting |
| `infrasynth.configs` | Typed per-tenant configuration values (global default + tenant override), Fernet-encrypted secrets, `config_changed`/`config_reset` signals |
| `infrasynth.billing` | App catalog, plans, entitlements, subscriptions, invoices, gateways, entitlement lifecycle jobs |
| `infrasynth.files` | Storage abstraction (S3/GCS/local/Cloudinary), signed URLs, processing pipelines, pluggable virus scanning |
| `infrasynth.notifications` | Multi-channel delivery with failover, retries, rate limits, and log retention |
| `infrasynth.webhooks` | Outbound delivery with HMAC + retry, verified inbound processing, event registry |
| `infrasynth.workflows` | State-machine engine, voting/approval strategies, validators |
| `infrasynth.scheduler` | Celery job dashboard + on-demand execution |

**Every app is multi-tenant.** One deployment, one schema, row-level isolation via a non-null `tenant_id`, a fail-closed `TenantManager`, and cross-tenant access that returns `404`. See `../TENANCY.md` (binding).

**There is no license server.** Access is an in-process *entitlement* (`is_entitled` / `check_limit`), enforced server-side. See `../ENTITLEMENTS.md` (binding).

---

## Configuration

Every knob is a namespaced dict with a safe default, read through `infrasynth.shared.settings_utils.get_setting`:

```python
INFRASYNTH_SECURITY = {"COOKIE_SECURE": True, "IP_BLACKLIST_THRESHOLD": 100}
INFRASYNTH_TENANCY = {"REQUIRE_TENANT_BY_DEFAULT": True, "TENANT_CLAIM": "tenant"}
INFRASYNTH_BILLING = {"GRACE_PERIOD_DAYS": 5, "DEFAULT_CURRENCY": "USD"}
INFRASYNTH_NOTIFICATIONS = {"CHANNELS": {"email": {"primary": "myapp.channels.SlackChannel"}}}
```

The full, commented reference lives in `config/settings/base.py`. Real settings files **never** live in this package — apps supply their own and pin the kit.

---

## Extending without forking

Integrate through settings, signals, registries, ABCs, and feature flags — never a local patch:

```python
# myapp/channels.py
from infrasynth.notifications.channels.base import BaseChannel
from infrasynth.shared.results import Result

class SlackChannel(BaseChannel):
    channel_type = "slack"
    def send(self, recipient, subject, body, is_html=True, attachments=None):
        ...
        return Result.ok(True)
    def health_check(self) -> bool:
        return True
    @classmethod
    def from_config(cls, config): return cls(**config)
```

Registries are populated in `apps.py:ready()`: `FeatureRegistry`, `PermissionRegistry`, `EventRegistry`, `VariableResolverRegistry`, `DataValidatorRegistry`, `PipelineStepRegistry`.

### Extension points, per module

Every module is swappable through settings/registries — no kit edits, no forks:

| Module | How a consumer extends it |
|---|---|
| `shared` | Use the primitives directly (`Result`, `encrypt/decrypt`, `get_setting`, protocols) |
| `api` | Override `renderer/pagination/exception handler` per view; add idempotency with `@idempotent` |
| `tenancy` | `TenantService`, scoped managers, `tenant_context`; configure `INFRASYNTH_TENANCY` |
| `security` | `AUTH_BACKEND_CLASS`; `TWO_FACTOR_SERVICE`/`TWO_FACTOR_RECOVERY_SERVICE`; `PasswordPolicyValidator`; custom permission classes; `infrasynth.gates`; `PermissionRegistry` |
| `audit` | `EXCLUDED_MODELS`/`EXCLUDED_FIELDS`/`SENSITIVE_KEYS`/`STORE_IN_DB`; listen to `security_event_occurred` |
| `features` | `FeatureRegistry.register(...)` or `INFRASYNTH_FEATURES["FLAGS"]`; `FeatureFlagOverride` rows |
| `configs` | `ConfigRegistry.register(...)` or `INFRASYNTH_CONFIGS["DEFINITIONS"]`; listen to `config_changed`/`config_reset` |
| `billing` | Gateway via `PaymentGateway.gateway_class`; `INVOICE_PDF_BUILDER`; `Entitlement`/`plan` |
| `files` | `register_storage_backend(...)` or `STORAGE_BACKENDS[name]["CLASS"]`; `PipelineStepRegistry.register(...)`; `PIPELINE_EXECUTOR`; `VIRUS_SCANNER` |
| `notifications` | `INFRASYNTH_NOTIFICATIONS["CHANNELS"]` dotted paths; `VariableResolverRegistry` |
| `webhooks` | `EventRegistry`; `InboundEndpoint.handler` dotted path; signature algorithm setting |
| `workflows` | `DataValidatorRegistry`; node/transition data; `WorkflowAwareModel` |
| `scheduler` | `ScheduledTask.task_path` dotted path (any Celery task or callable) |

A consuming app never imports another app's models directly — it uses the
services, registries, signals, and `infrasynth.gates` documented here.

---

## Tenant configuration

`infrasynth.configs` is a generic, typed, per-tenant configuration store for
scalar/JSON preferences (branding, limits, integration settings). Precedence is
**tenant override → global default → registry default**; reads fail closed, so
without a tenant context only the global row and the registry default are visible.

```python
# myapp/apps.py → ready(): declare the key (or set INFRASYNTH_CONFIGS["DEFINITIONS"])
from infrasynth.configs import ConfigRegistry
ConfigRegistry.register("branding.primary_color", type="string", default="#1a3a5c", group="branding")

# read / write
from infrasynth.configs import ConfigService
ConfigService().get("branding.primary_color")            # tenant override, else global, else default
ConfigService().set("branding.primary_color", "#0044cc", tenant=tenant, user=request.user)
ConfigService().set_global("branding.primary_color", "#1a3a5c")   # platform default
```

- **Secrets:** mark a definition `is_secret=True`; the value is Fernet-encrypted
  at rest and masked (`None`) in the API, signals, and audit payloads.
- **API:** `GET /api/v1/configs/` (effective values, `?group=`/`?keys=`),
  `GET/PUT/DELETE /api/v1/configs/<key>/`, `GET /api/v1/configs/definitions/`,
  and `PUT /api/v1/configs/global/<key>/`. Writes require `configs.manage`
  (tenant) / `configs.manage_global` (platform), with the owner bypass.
- **Cross-tenant:** the key is always resolved for the request tenant; another
  tenant's value is never returned, and an unknown key returns `404`.

---

## Permissions & roles (automatic)

Every model gets permissions automatically, and assigning them to roles/users
through the API enforces them without touching code.

**Derived codenames** (`{app_label}.{verb}_{model}`, Django-style):

| Action | Codename |
|---|---|
| `list` / `retrieve` | `{app}.view_{model}` |
| `create` | `{app}.add_{model}` |
| `update` / `partial_update` | `{app}.change_{model}` |
| `destroy` | `{app}.delete_{model}` |

Subclass the kit base viewset and enforcement is automatic:

```python
# helpdesk/views.py — no required_permissions needed
from infrasynth.security.viewsets import InfraSynthModelViewSet

class TicketViewSet(InfraSynthModelViewSet):
    queryset = Ticket.objects.all()
    serializer_class = TicketSerializer
    # requires helpdesk.view/add/change/delete_ticket
    action_permissions = {"resolve": "helpdesk.resolve_ticket"}  # optional, for @action

    @action(detail=True, methods=["post"])
    def resolve(self, request, pk=None): ...
```

- **Custom permissions** live in *your* code (never the kit):
  `PermissionRegistry.register("helpdesk.export_ticket", name="Export tickets", group="Helpdesk")`
  in your `apps.py:ready()`.
- **Enforcement modes:** `INFRASYNTH_SECURITY["AUTO_PERMISSIONS"]` is `"global"`
  (default; model-backed views are gated everywhere, non-model views abstain),
  `"opt_in"` (only kit base viewsets / `auto_permissions = True`), or `"off"`.
  An explicit `required_permissions` always wins; it is **any-of** unless the
  view sets `require_all = True`. Tenant owners and superusers bypass.
- **Catalog:** `manage.py sync_permissions` (also runs on `post_migrate`) upserts
  every derived + custom codename into `security_permission`; `GET
  /api/v1/auth/permissions/` lists it (filter with `?app_label=`/`?group=`) for
  building the assignment UI. Unknown codenames are rejected on write when
  `STRICT_PERMISSION_VALIDATION` is on.
- **Assignment:** tenant roles (`POST /api/v1/auth/roles/`) are per-tenant and
  assigned per user with `/api/v1/auth/users/<id>/roles/`; global roles
  (`tenant IS NULL`, require `platform.roles.manage`) apply in every tenant.
  `Grant`/`Revoke` add per-user overrides — tenant-scoped by default, or
  platform-wide with `{"scope": "global"}` (requires `platform.roles.manage`).

### Signals

Every kit signal carries `tenant_id` explicitly (a global write uses
`tenant_id=None`, `scope="global"`). Consumers connect in `apps.py:ready()`:

```python
# myapp/apps.py → ready()
from infrasynth.configs import config_changed

def on_config_changed(sender, tenant_id, key, scope, old_value, new_value, actor_id, **kwargs):
    if key == "branding.primary_color":
        refresh_theme_cache(tenant_id)

config_changed.connect(on_config_changed, dispatch_uid="myapp.theme")
```

| Signal | Emitted when | Key kwargs |
|---|---|---|
| `configs.config_changed` | a tenant/global value is written | `tenant_id`, `key`, `scope`, `old_value`, `new_value`, `actor_id` (secrets masked) |
| `configs.config_reset` | a tenant override is deleted | `tenant_id`, `key`, `scope`, `previous_value`, `actor_id` |
| `features.flag_created` / `flag_toggled` / `flag_deleted` | a feature flag is created/toggled/deleted | `tenant_id`, `flag_slug`, `is_active`, … |
| `features.override_created` / `override_deleted` | a user/group flag override changes | `tenant_id`, `flag_slug`, `user_id`, `is_enabled` |
| `tenancy.tenant_updated` | a tenant's editable fields change | `tenant_id`, `changes`, `actor_id` |
| `scheduler.task_completed` / `task_failed` | a task execution reaches a terminal status | `tenant_id`, `task_name`, `task_id`, … |
| `audit.model_changed` | a tracked model create/update/delete is logged | `tenant_id`, `model_label`, `object_id`, `action`, `changes` |

---

## Gating your own endpoints (no kit edits)

`infrasynth.gates` is the composable, per-endpoint access layer. A view declares
exactly what it needs; the default is **gate nothing**:

```python
from rest_framework.permissions import AllowAny, IsAuthenticated
from infrasynth.gates import (
    GatePermission, gated,
    TwoFactorGate, AltchaGate, EntitlementGate, FeatureGate, PermissionGate,
)

# Public form: proof-of-work, no login.
class SignupView(APIView):
    permission_classes = [AllowAny, GatePermission]
    infrasynth_gates = [AltchaGate()]

# Sensitive action: step-up 2FA + commercial right + codename.
class PayoutView(APIView):
    permission_classes = [IsAuthenticated, GatePermission]
    infrasynth_gates = [
        TwoFactorGate(),                                # passes users without 2FA;
        EntitlementGate("billing", feature="payouts"),  # use require_configured=True to demand setup
        PermissionGate("billing.payout"),
    ]

class TicketViewSet(ModelViewSet):
    permission_classes = [IsAuthenticated, GatePermission]
    infrasynth_gates = [FeatureGate("ticketing")]       # 404 when the flag is off

    @gated(TwoFactorGate())
    @action(detail=True, methods=["post"])
    def close(self, request, pk=None): ...
```

- Gates run in order; the first denial raises the matching namespaced error
  (`AUTH_*`, `ENTITLEMENT_*`, `VALIDATION_*`, `NOT_FOUND`) so the envelope gets
  the right code and status. No gates declared ⇒ the permission is a no-op.
- `GatePermission` is in `DEFAULT_PERMISSION_CLASSES`; the kit's own
  `HybridPermission` (aliased `IsAuthenticatedAndPermitted`) also evaluates
  declared gates, so you only add it explicitly on views that use plain DRF
  permissions.
- `AltchaGate` accepts the solution in a JSON `altcha` object, flat body/query
  keys, or the `X-Altcha: <challenge_id>:<solution>:<number>` header; clients get
  a challenge from `/api/v1/auth/altcha/challenge/`.
- `TwoFactorGate` verifies a JWT `2fa` claim (minted at verification and carried
  across workspace selection) or a verified session. Users without 2FA pass by
  default; `require_configured=True` denies them with `AUTH_2FA_SETUP_REQUIRED`.

Every gate is also a plain class implementing `check(request, view) -> GateResult`,
so an app can ship its own (e.g. an IP allow-list) and pass it to `@gated(...)`.

---

## Stable import paths

Consumers import from the public modules, never internal helpers:

| Need | Import |
|---|---|
| Gating | `from infrasynth.gates import GatePermission, AltchaGate, …` |
| Permissions/auth | `from infrasynth.security.permissions import HybridPermission` |
| Automatic permissions | `from infrasynth.security import InfraSynthModelViewSet, PermissionRegistry, AutoPermission` |
| JWT/API-key auth | `from infrasynth.security.auth.cookies import CookieJWTAuthentication` |
| Authorization | `from infrasynth.security.services import AuthorizationService` |
| Tenant context/scoping | `from infrasynth.tenancy.managers import TenantManager` |
| Entitlements | `from infrasynth.billing.entitlements import EntitlementService` |
| Feature flags | `from infrasynth.features.services import FeatureService` |
| Configuration | `from infrasynth.configs import ConfigService, ConfigRegistry, config_changed` |
| Storage | `from infrasynth.files.storage import get_storage_backend` |
| Errors/envelope | `from infrasynth.shared.exceptions import EntitlementError` |
| Wire format | `from infrasynth.api.renderers import EnvelopeJSONRenderer` |


---

## Scheduled work

`CELERY_BEAT_SCHEDULE` ships with the kit: notification retries/log purge, billing sync + lifecycle + renewal invoices, and audit retention. Run `celery -A config beat` and `celery -A config worker`.

---

## Quality bar

- `ruff check` + `ruff format --check` + `mypy infrasynth/` are CI gates.
- `pytest` runs the full suite; coverage is enforced in CI (see `pyproject.toml`).
- A change to kit behavior belongs **here**, then consuming apps bump their pin — see `../AGENTS.backend-packages.md` §9.