gdegelas/solar
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
main
solar/docs/GENERATION_WORKFLOW_ARCHITECTURE.md

20 KiB
Raw Permalink Blame History

Generation Workflow – System Architecture (Deep-Dive)

Product: Solar Production Forecasting (degelas) – degelas.be · PR: pr.degelas.be
Scope: Optimal and advanced system architecture for the post generation workflow so that all PR/marketing posts are grounded in the real product (frontend features, backend data, and docs). Posts promote Solar Production Forecasting and related capabilities; generation uses this project as the single source of truth.

1. Product context (what posts are about)

1.1 Brand and product

Item Value
Product name Solar Production Forecasting
Tagline Happy Days!
Brand / domain degelas (degelas.be, www.degelas.be)
API Solar Forecast API (backend); v1.4.5

All generated posts should reference this product and, where useful, the public site (degelas.be) or specific features.

1.2 Frontend features (from frontend project)

These are the user-facing capabilities that posts can promote. Generation templates and AI briefs should align with these names and value props.

Feature Path (example) Value for posts
Dashboard /{country} e.g. /be Solar + weather metrics, 3 days past + 14 days forecast, estimated kWh, location cards
Hourly /hourly Hour-by-hour metrics by location
Compare /compare Side-by-side locations
Prices /prices Day-ahead electricity prices (€/MWh), zones, refresh
EU History /euhistory Historical EU energy prices, zones, date range
Trading /trading Production value, backtest at day-ahead price
Battery trading /batterytrading Battery arbitrage: charge cheap / discharge expensive
EV charging /evcharging EV smart charging vs day-ahead prices, savings vs baseline
Tibber /tibber Tibber integration (prices/usage)
Weekly planner /weeklyplan Weekly plan: EV + prices + optional solar
Real-life use case /usecase Scenario-based personas (family, couple, etc.)
Deep Dive /deepdive Analytics: production/temperature rankings, price surges, volatility
Solar clock /solarclock World solar clock: sunrise/sunset, radiation, estimated kWh by location/date
Log /log Activity log

Implications for generation:
Templates and AI prompts should use these feature names and paths (e.g. “Battery trading”, “EV smart charging”, “Solar clock”, “Deep Dive”) and link to degelas.be with the right path (e.g. /trading, /evcharging, /solarclock) so posts drive traffic to real product surfaces.

1.3 Backend domains (from backend/app)

These are data and API surfaces the generation workflow can use (read-only) to keep copy accurate and, optionally, data-driven.

Domain Key APIs / data Use in generation
Config GET /config Solar (panel count, kWp, PR), battery (kWh, kW), EV defaults → product setup in copy
Countries & locations GET /countries, GET /locations?country= Location names, countries → “Brussels”, “Belgium”, “Europe” in posts
Metrics GET /metrics, GET /metrics/all, GET /history, GET /hourly, GET /metrics/solar-clock Estimated kWh, solar, weather → data-driven one-liners (e.g. “Today’s estimate: X kWh”)
Prices GET /prices, GET /prices/pattern, GET /prices/compare Day-ahead prices, cheap/expensive hours → “Charge when it’s cheap” angles
EU History GET /eu-history/prices, GET /eu-history/compare Historical prices → “Compare with last year” angles
Trading GET /trading/backtest, GET /trading/expected, POST /trading/ev-schedule, weekly plan, scenarios, battery backtest Backtest, expected earnings, EV savings, scenarios → “See your savings” angles
Products GET /products (solar, battery, ev) Product templates (CAPEX, kWp, capacity) → “From X kWp solar to Y kWh battery”
Scenarios GET /trading/scenarios, scenario preview/optimized Personas (family, couple) → “Real-life scenarios” angles
Deep Dive GET /deepdive Rankings, surges, volatility → “Explore the data” angles

Implications:

  • Static context: Config, locations, products, scenarios can be loaded once (or cached) and fed into content templates or AI system prompts.
  • Optional live data: For “today’s solar” or “this week’s cheap hours”, the generator can call internal backend APIs (or a thin “generation context” service that calls them) and inject numbers into templates or AI context. This is optional; templates can also use placeholders like {{location_name}} or {{product_kwp}} filled from config/products.

1.4 Docs (from docs/)

Doc Role in generation
PR_MARKETING_ARCHITECTURE.md Campaign/post CRUD, workflow (generate → DB → degelas-chrome), templates vs AI, platform limits
AVAILABLE_DATA.md What we store (solar, weather, AQ, weekday/weekend, etc.) → data angles for copy
TRADING_PLATFORM.md, BATTERY_TRADING.md Trading and battery value props → accurate wording
DEEPDIVE_ANALYSIS_PLAN.md, REAL_LIFE_SCENARIO_REVIEW.md Analytics and scenarios → feature descriptions
BROWSER_API.md How posting works (X/Instagram via degelas-chrome) – operational, not content
DATA_SOURCES.md, CRON.md, HISTORY_AND_LIMITS.md Data sources and jobs – for “what we can say” and freshness

Implication:
Generation (templates + AI) should be informed by these docs so claims (e.g. “battery arbitrage”, “EV smart charging”, “solar production forecast”) match the real product and data.

2. High-level workflow (unchanged)

The pipeline remains:

  1. Generate – Produce one or more post texts (and optionally scheduled_at, platform, campaign_id) using product context (frontend features, backend config/data, docs).
  2. Validate – Enforce platform rules (X 280, Instagram 2200, max N posts per platform per day); backend enforces on create/update.
  3. Persist – POST /scheduled-posts or POST /scheduled-posts/bulk (existing APIs). DB is source of truth.
  4. Publish – Existing scheduler job + do_browser_post + degelas-chrome (no change).

So: generation is the only new “system”; the rest is existing backend + PR frontend + degelas-chrome.

3. Optimal system architecture for generation

3.1 Layered view

┌─────────────────────────────────────────────────────────────────────────────┐
│  LAYER 1: Product & content model                                            │
│  • Product name, tagline, base URL (degelas.be)                              │
│  • Feature catalog: names, paths, one-line value props (from frontend)       │
│  • Content variables: locations, products, scenarios (from backend/config)  │
│  • Optional: live context (today’s metric, this week’s cheap hours)         │
└─────────────────────────────────────────────────────────────────────────────┘
                                        │
                                        ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│  LAYER 2: Schedule model                                                     │
│  • Campaign window (start_date, end_date)                                   │
│  • Days of week (e.g. Mon–Fri, weekends only)                               │
│  • Time slots (UTC or local), e.g. 09:00, 12:00, 18:00                       │
│  • Platform split per day (e.g. 2 X + 1 Instagram)                          │
│  • Respects max_posts_per_platform_per_day (global + campaign override)      │
└─────────────────────────────────────────────────────────────────────────────┘
                                        │
                                        ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│  LAYER 3: Generation service                                                 │
│  • Content templates: e.g. "{{feature_name}}: {{value_prop}} – {{url}}"      │
│  • Variable resolution: from Layer 1 (and optional live API calls)           │
│  • Optional: AI (OpenAI) with system prompt = product context + limits       │
│  • Output: list of { platform, text, scheduled_at, campaign_id? }           │
└─────────────────────────────────────────────────────────────────────────────┘
                                        │
                                        ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│  LAYER 4: Validation & persistence (existing)                              │
│  • POST /scheduled-posts or /scheduled-posts/bulk                           │
│  • Backend enforces length + posts-per-day; returns created posts            │
└─────────────────────────────────────────────────────────────────────────────┘
                                        │
                                        ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│  LAYER 5: Publish (existing)                                                 │
│  • Scheduler job → get_due_scheduled_posts → do_browser_post → degelas-chrome│
└─────────────────────────────────────────────────────────────────────────────┘

3.2 Data flow (end-to-end)

  1. Inputs

    • Campaign (name, start_date, end_date, optional max_posts override).
    • Schedule template (days_of_week, time_slots_utc, platform_slots).
    • Content source: either content template (e.g. Mustache/Handlebars) with variables from Layer 1, or AI request (brief + product context from Layer 1).

  2. Expand schedule (Layer 2 → slot list)
    For each day in [start_date, end_date]:

    • If day in days_of_week: for each time_slot and each platform according to platform_slots, emit (date, time_utc, platform).
    • Cap per platform per day by max_posts (global/campaign).
      Result: list of (date, time_utc, platform) slots.

  3. Generate copy (Layer 3)
    For each slot (optionally batched):

    • Template path: resolve variables (feature_name, value_prop, url, location_name, product_kwp, …) from Layer 1; optional live call (e.g. “today’s estimated_kwh for Brussels”); render template → text.
    • AI path: build user message (e.g. “One tweet for EV smart charging, max 280 chars, link degelas.be/evcharging”); system prompt = product context + platform limits; call OpenAI → text.
      Output: list of (platform, text, scheduled_at) with optional campaign_id.

  4. Persist (Layer 4)
    POST /scheduled-posts/bulk with campaign_id. Backend validates and writes; job (Layer 5) later posts via degelas-chrome.

3.3 Where the “product” lives (Layer 1)

To keep generation based on this project and consistent:

  • Single product config (code or DB)

    • Product name, tagline, base URL.
    • Feature catalog: for each frontend feature, store: name (e.g. “Battery trading”), path (e.g. /batterytrading), short value prop (e.g. “Charge when cheap, discharge when expensive”).
    • Can be derived from frontend routes + i18n or maintained in backend (e.g. GET /config/features or a small table).

  • Variables for templates

    • From backend: GET /config (solar_kwp, battery_kwh, …), GET /locations, GET /products, GET /trading/scenarios.
    • Optional: cached or on-demand “today’s metric for location X”, “cheap hours this week” (from prices/pattern or trading).

  • Docs as reference

    • Use AVAILABLE_DATA, TRADING_PLATFORM, BATTERY_TRADING, REAL_LIFE_SCENARIO_REVIEW, etc., to write system prompts or template library text so generated copy never contradicts the product.

This makes the frontend project (and backend + docs) the source of truth for “what we offer” and “what we can say”.

4. Advanced behaviour (multiple campaigns, time-of-day, days-of-week)

  • Multiple campaigns
    Each campaign has its own window and optional schedule template. The same generation service can expand many campaigns; each expansion produces a set of slots and then copy. Posts are tagged with campaign_id so reporting and calendar stay per-campaign.

  • Time-of-day and days-of-week
    Handled entirely in Layer 2 (schedule model):

    • days_of_week: e.g. [0,1,2,3,4] = Mon–Fri; [5,6] = weekend.
    • time_slots_utc: e.g. ["09:00", "12:00", "18:00"] (interpret in UTC or convert from local).
    • Different campaigns can have different schedules (e.g. “morning only” vs “evening only” vs “weekdays 09 and 18”).

  • Platform split
    In schedule template: e.g. “2 X + 1 Instagram per day”. Expansion allocates slots to platforms respecting that split and max per platform per day.

  • Adjustable in the template
    “Template” here means both:

    • Schedule template (when to post, which platform) – adjustable per campaign.
    • Content template (what to say) – variables filled from product context; can vary by campaign or by slot (e.g. “Monday = solar, Wednesday = EV”).

5. AI (OpenAI) – when and how

  • When it helps

    • Many variants (e.g. “10 tweet ideas for EV feature”).
    • Tone/audience (e.g. “professional” vs “friendly”).
    • One-off or A/B copy without maintaining a big template set.

  • How to keep it product-based

    • System prompt = fixed product context: product name, tagline, list of features (name + path + one-line value prop), platform limits (X 280, Instagram 2200), and “only mention features that exist at degelas.be”.
    • User prompt = slot/campaign brief: e.g. “One tweet for feature ‘EV smart charging’, include link to degelas.be/evcharging, max 280 characters.”
    • Output = text only; creation still goes through Layer 4 (create/bulk) so validation and DB stay the same.

  • Optional live data in AI context

    • E.g. “Today’s estimated production for Brussels: X kWh” from GET /metrics or history.
    • Injects real numbers into AI context so posts can be data-driven without hardcoding.

6. Implementation phases (recommended order)

  1. Product & content model (Layer 1) ✅

    • Define product name, base URL, tagline in one place (config or DB).

    • Build feature catalog from frontend (names, paths, one-line value props).

    • Expose as e.g. GET /config/product-context or static JSON used by the generator.

    • Optionally: small “context” service that calls GET /config, GET /locations, GET /products, GET /trading/scenarios and returns a single payload for templates/AI.

    • Implemented: GET /config/product-context (PR auth); app/product_context.py + app/product_context_api.py; optional ?include_config=1 for solar/battery config.

  2. Content templates (Layer 3, template path)

    • Add a small template engine (e.g. Mustache/Handlebars or Python string.Template) and a template store (DB or files).
    • Variables: {{product_name}}, {{base_url}}, {{feature_name}}, {{feature_path}}, {{value_prop}}, {{location_name}}, etc., filled from Layer 1.
    • “Generate from template” in PR UI: pick template + campaign (and optional schedule), resolve variables, preview, then call create/bulk.

  3. Schedule template (Layer 2) ✅

    • Model: days_of_week, time_slots_utc, platform_slots. Implemented: app/schedule_templates.py; GET /generate/schedule-templates, POST /generate/expand, POST /generate/expand-and-create.
    • Expand endpoint: given campaign_id (or window) + schedule template id, compute (date, time_utc, platform) slots; optionally generate text per slot from content template; call POST /scheduled-posts/bulk.
    • PR UI: “Expand campaign from schedule” so one click fills the calendar within limits.

  4. Optional AI (Layer 3, AI path)

    • Endpoint e.g. POST /generate/suggest-posts: body = campaign_id or window, platform, count, optional brief; system prompt = product context; return suggested texts only.
    • PR UI: “Suggest posts” → show list → user edits → “Create selected” → create/bulk.
    • Optional: “Generate for this slot” using same system prompt + slot brief.

  5. Optional live data

    • For “today’s solar” or “cheap hours”: generator (or context service) calls GET /metrics, GET /prices/pattern, etc.; injects into template vars or AI context.
    • Start with one or two hooks (e.g. one location’s today estimate, one zone’s cheap hours) to keep scope small.

  6. No change to Layer 4 (existing create/bulk) or Layer 5 (scheduler + degelas-chrome).

7. Summary

  • Product: Solar Production Forecasting (degelas) – all frontend features (dashboard, hourly, compare, prices, EU history, trading, battery, EV, Tibber, weekly planner, use case, Deep Dive, solar clock) and backend (config, locations, metrics, prices, trading, products, scenarios) form the source of truth for what posts talk about.
  • Architecture: Five layers – (1) product & content model, (2) schedule model, (3) generation service (templates + optional AI), (4) validation & persistence (existing APIs), (5) publish (existing job + degelas-chrome).
  • Advanced: Multiple campaigns, time-of-day, days-of-week, platform split, and “adjustable in the template” are handled in Layers 1–3; posting and limits stay in Layers 4–5.
  • Implementation: Start with product/feature catalog and content templates grounded in the frontend/backend; add schedule expansion; then optional AI and optional live data. This yields an optimal and advanced generation workflow based on this given project (frontend, backend, docs) while reusing the existing pipeline to the database and degelas-chrome.