Vergangene Daten aus dem HA-Verlauf importierbar, Auslieferstand bereinigt

Datenaufbewahrung (recorder_snippet.yaml, neu): Home Assistant loescht
Sensor-Verlaeufe standardmaessig nach 10 Tagen - das war die stille
Obergrenze dafuer, wie weit sich ueberhaupt je etwas rekonstruieren
laesst, denn geloeschte recorder-Zeilen sind endgueltig weg. Der Block
hebt das auf 365 Tage. Die Bestaende der App selbst (fahrten.jsonl,
tankvorgaenge.jsonl, batteriespannung.jsonl, fahrzeugprofil.json) waren
davon nie betroffen, die kennen ohnehin keine Purge-Logik. Platzbedarf
an der Testinstanz gemessen statt geschaetzt: 90.858 Zeilen in 17 Tagen,
hochgerechnet grob 1,9 Mio. Zeilen und 1-1,5 GB im Jahr, davon 96 % aus
der EU-Data-Act-Integration - dafuer liegt eine auskommentierte
exclude:-Liste bei. Bewusst kein include:-Block, der wuerde jeder
anderen Integration den Verlauf nehmen und dabei kaum Platz sparen.

Historienimport (pyscript/historienimport.py, neu): neuer Dienst
audi_dashboard_historie_importieren(start, ende) liest denselben
Verlauf, den die Live-Trigger in Echtzeit sehen, und leitet rueckwirkend
dieselben Datensaetze ab - Fahrten aus dem Zuendungsverlauf (inklusive
der Pausenregel, kurze Unterbrechungen verschmelzen zu einer Fahrt),
Tankvorgaenge nach derselben Tiefststand-Logik und denselben Schwellen
wie tankerkennung.py, Tagesmin/-max der Batteriespannung. Mehrfach
ausfuehrbar: ueberschneidet sich eine Fahrt mit einer bereits erfassten,
wird sie uebersprungen statt doppelt angelegt. Erzeugte Datensaetze
tragen source: "import".

Gelesen wird ueber HAs eigene recorder-API (get_significant_states),
nicht per direktem SQL auf home-assistant_v2.db: das Schema ist
HA-intern und aendert sich zwischen Versionen, die Funktion ist die
stabile Schnittstelle. Preis dafuer ist hass_is_global: true im
pyscript-Block; ohne die Zeile laeuft alles andere unveraendert weiter,
nur der Import meldet, dass er den Verlauf nicht lesen kann.

Oberflaeche in beiden Codebasen: Knopf "Daten importieren aus Home
Assistant" unter Einstellungen -> Einrichten, dahinter ein Fenster mit
Von/Bis (Vorbelegung: letzte 30 Tage), "Importieren" als Hauptaktion und
"Abbrechen" als zweite Wahl. Das Fenster bleibt offen und zeigt das
Ergebnis, statt optimistisch zu schliessen - hier ist das Ergebnis der
Zweck des Aufrufs. Fortschritt ueber die Entitaet
pyscript.audi_dashboard_import_status, weil pyscript-Dienste sofort
zurueckkehren. In companion-app bewusst NICHT ueber die
Offline-Warteschlange: ein spaeter aus dem Nichts abgefeuerter
Importlauf waere fuer den Nutzer nicht nachvollziehbar.

Beim Testen gefunden und mitgefixt: profil.batterieverlauf_tageswert_
aktualisieren() haengte in Einfuegereihenfolge an. Solange nur die
Live-Aufzeichnung schrieb, war das dasselbe wie sortiert (sie traegt
immer den heutigen Tag ein) - der Import traegt vergangene Tage nach,
die waeren hinter den neueren gelandet und das Diagramm haette zeitlich
rueckwaerts gelaufen. Sortiert jetzt nach Datum, wie der eigene
Docstring es ohnehin versprach.

Auslieferstand bereinigt: einstellungen.py brachte zwei Entity-IDs einer
laengst abgeraeumten Testinstanz mit (testzone_fmm003_...). Beim Testen
unsichtbar, weil die Testinstanz sie ueber entitaeten.json ueberschreibt
- auf einer Neuinstallation waeren sie die wirksamen Werte gewesen. Und
schlimmer als nur wirkungslos: fahrterkennung.py registriert seinen
@state_trigger nur, wenn das Feld belegt ist, ausdruecklich als Schutz
gegen eine leere Entity-ID. Ein gesetzter, aber nicht existierender Wert
hebelt genau diesen Schutz aus. Beide Felder jetzt leer wie alle
anderen; Kopfkommentar der Datei ueberarbeitet, er behauptete noch, die
EU-Data-Act-Integration werde nicht mehr verwendet.

install.ps1 abgesichert, da die erste echte Installation auf HA OS
ansteht: die feste configuration.yaml.bak wurde bei jedem Lauf
ueberschrieben, ausgerechnet das Original war nach dem zweiten Lauf weg
- Sicherungen jetzt zeitgestempelt. Nach dem Schreiben wird die Datei
zurueckgelesen und geprueft (Laenge, genau ein Markerblock, bisheriger
Inhalt unveraendert); bei der kleinsten Abweichung rollt das Skript
automatisch zurueck. Die Sicherheitseigenschaften stehen jetzt im Kopf
der Datei, statt dass man ihnen glauben muss.

Standort-Blatt: "Teilen" war im Tagmodus unsichtbar - .standort-pille
nutzte var(--tile), das Blatt darunter var(--tile-deckend), und die
iOS-Auflage setzt im Tagmodus beide auf #FFFFFF. Gemessener Kontrast
1,00:1, weiss auf weiss. Beide Pillen folgen jetzt der Knopfsprache des
Panels (.aktion / .aktion.primaer): Umriss fuer die Nebenaktion, var(--fg)
gefuellt fuer "Route". Danach 17-21:1 Textkontrast in beiden Themes.
Nebenbei ist damit --line-strong - eine Linienfarbe - nicht laenger als
Knopffuellung im Einsatz.

Geprueft: Import zweimal ueber die echte Oberflaeche im Browser gegen
eine in die recorder-DB eingespielte Kunsthistorie (der Testcontainer
laeuft nicht, waehrend das Auto faehrt, echte Fahrten liegen dort also
nicht vor) - drei Fahrten wie erwartet, die 5-Minuten-Unterbrechung
korrekt zu einer 50-Minuten-Fahrt verschmolzen, die 30-Sekunden-Zuendung
verworfen, Strecken kilometergenau; zweiter Lauf legte 0 an und meldete
3 als vorhanden. Neuinstallation in einem Wegwerf-Container: nur
Profilvorlage und Parser, keine Fahrten-/Tank-/Batteriedatei, null
Fehler im Log. install.ps1 gegen Attrappen: -Pruefen schreibt nichts,
echter Lauf laesst automations.yaml bytegleich (SHA-256) und fremde
Bloecke stehen, Ergebnis parst als gueltiges HA-YAML, zweiter Lauf
idempotent, bei fremdem pyscript:/panel_custom: bleibt die Datei
bytegleich. tsc --noEmit sauber, companion-app-Tests 106/106, vite build
sauber, HA-Configcheck und Start ohne Fehler.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

AGENTS.md

@@ -1,6 +1,6 @@
 # AGENTS.md — Project state, review findings, open items, and working rules
 
-**Last updated: 2026-08-19** (merged the `umsetzung-datametric360` branch — companion app phases
+**Last updated: 2026-08-23** (merged the `umsetzung-datametric360` branch — companion app phases
 1–10 done, see `UMSETZUNGSPLAN.md`; 2026-08-13: cleaned up remaining EU Data Act residue, fixed
 oversized toggle switches, and fixed a desktop-layout audit (settings button / rings logo / popups
 overflowing past the capped content column) — see `DESIGN_AUDIT_2026-08-13.md`; 2026-08-16: fixed
@@ -87,6 +87,11 @@ Verified live in the Docker test container (panel: add/edit/delete a Selbstbetei
 values persisted through profilSpeichern()+re-render) and via `tsc --noEmit` + the companion-app test
 suite (100/100 incl. the render-every-page smoke test, which now also covers `vertrag`) + `vite
 build`, since companion-app needs a live backend connection the local dev server doesn't have.
+2026-08-23: **retroactive data import** — see section F below for the full entry. In short: HA's
+recorder keeps sensor history only 10 days by default, which silently capped how far back anything
+could ever be reconstructed; `recorder_snippet.yaml` raises that to a year, and a new
+`pyscript/historienimport.py` + "Daten importieren aus Home Assistant" button (both codebases)
+rebuilds trips, refuels and battery history for any past window out of that recorder history.
 This file is the entry point for every new agent
 session: what this repo is, what is finished, what is missing, and how to work here. Detail lives in
 the linked documents — this file points, it does not duplicate.
@@ -1705,6 +1710,152 @@ Not planned/scoped in either codebase yet — start with clarifying questions on
 and the summary-box redesign goals before touching code, per this project's mockup-first convention
 for open-ended redesign asks (see the "Anstehende Termine" 3-round mockup process, section C).
 
+### F) Retroactive data import from the HA recorder (built 2026-08-23)
+
+**The retention answer, because it was the owner's actual question.** Two different lifetimes were
+being conflated:
+
+- **The app's own records live forever already.** `fahrten.jsonl`, `tankvorgaenge.jsonl`,
+  `batteriespannung.jsonl` and `fahrzeugprofil.json` under `/config/audi_dashboard/` have no purge
+  path anywhere in `profil.py` — nothing trims them, ever. That was already true before this change.
+- **HA's raw sensor history did not.** `recorder.purge_keep_days` defaults to **10 days**, and the
+  test instance had no `recorder:` block at all. That default was the real ceiling: it caps how far
+  back *any* reconstruction can ever reach, because deleted recorder rows are gone permanently.
+
+`homeassistant/recorder_snippet.yaml` (new) raises it to 365 days with `auto_repack: true`. Measured
+on the test instance rather than guessed: 90.858 state rows over 17 days ≈ 5.300/day, 64,7 MB DB →
+roughly 1,9 M rows and 1–1,5 GB for a year. 96 % of that traffic is the `cupra_eu_data_act`
+integration (~1.737 states per entity per 17 days across ~50 entities), 1 % flespi/FMM003, 2 % the
+rest — so the file carries a commented-out `exclude:` list targeting the EU-Data-Act side for anyone
+who needs to shrink it. Deliberately **no `include:` block**: an include would stop HA recording
+everything else entirely, and since 97 % of this instance is vehicle data it would save almost
+nothing while costing every other integration its history.
+
+**The import itself.** `pyscript/historienimport.py` (new) exposes
+`audi_dashboard_historie_importieren(start, ende)`. It reads the same sensors the live triggers
+watch and derives the same records they would have produced:
+
+- trips from the ignition sensor — contiguous "on" stretches, then merged where the gap is shorter
+  than `fahrten_pausenzeit_min`, which reproduces the live pause rule (`task.unique()` +
+  `task.sleep()` in `fahrterkennung.py`) after the fact; odometer, GPS start/end filled from the
+  other histories; stretches under 60 s dropped as ignition-without-a-drive
+- refuels from the tank sensor using the same low-water-mark logic and the same thresholds as
+  `tankerkennung.py` (5 l / 9 pp, whichever is more sensitive)
+- daily min/max battery voltage, same shape as `batterieverlauf.py`
+
+Re-running is safe by design: trips whose window overlaps an existing trip are skipped (whatever
+their origin), refuels dedupe on a 90-minute window, battery days merge in `profil`'s own
+min/max update. Created records carry `source: "import"` alongside the existing `ha`/`manual`/`auto`.
+
+**How the history is read, and why.** Via HA's own
+`homeassistant.components.recorder.history.get_significant_states` through `task.executor()`, not by
+opening `home-assistant_v2.db` with sqlite3 directly. Direct SQL was the tempting option
+(`allow_all_imports` is already on, no config change needed) but the recorder schema is HA-internal
+and changes between versions; the function is the interface HA itself uses. The cost is one new
+config line, **`hass_is_global: true`** in the pyscript block (`configuration_snippet.yaml`), without
+which the `hass` name does not exist in pyscript — everything else in the app works fine without it,
+only the import reports that it cannot read the history. `significant_changes_only=False` matters:
+the default mode drops exactly the small numeric steps that odometer distance and refuel detection
+are built from.
+
+**Frontend, both codebases.** Panel: `vImportPopup()` reusing the existing `.beleg-popup` overlay
+pattern, opened from a new button under Einstellungen → Einrichten, with Von/Bis
+`datetime-local` fields defaulting to the last 30 days, "Importieren" primary and "Abbrechen"
+second. The dialog stays open and shows the result rather than closing optimistically — the one
+place in the app where that is right, because the result *is* the point of the action. Progress
+comes from polling `pyscript.audi_dashboard_import_status` (same pattern as the update status),
+since pyscript services return immediately while the work continues in the background.
+Companion: `screens/HistorienImport.tsx` using the design system's `Popup variant="form"`, plus
+`api.historieImportieren()` / `api.importStatusLesen()`. Deliberately **not** routed through the
+offline queue like every other write: the import is a long-running operation whose result is the
+reason you invoked it, and a queued run firing later from a dead session would surprise the user —
+offline, it simply should not be offered.
+
+**One real bug found and fixed while testing:** `profil.batterieverlauf_tageswert_aktualisieren()`
+appended in insertion order. That was invisible while only the live path wrote it (it always adds
+today, i.e. always the newest), but the import adds *past* days — they landed after newer ones and
+the frontend chart, which draws in file order, would have run backwards. It now sorts by date,
+restoring the invariant its own docstring already promised.
+
+**Verification.** The test container is not running while the car is driven (owner's note), so its
+recorder holds no real trips. Instead a synthetic history was seeded straight into the recorder DB —
+two drives, one with a 5-minute mid-drive ignition gap, one 30-second ignition burst, a two-step
+refuel, and two days of voltage — and the import reproduced it exactly: 3 trips (the gap correctly
+merged into one 50-minute/40 km trip), 1 rejected as too short, 1 refuel, 2 battery days, distances
+matching the seeded odometer to the kilometre. A second run over the same window created 0 and
+reported 3 as already present, confirming dedup. Both runs were driven through the real UI in the
+browser, not by calling the service directly. `tsc --noEmit` clean, companion tests 106/106 (6 new
+for the result-sentence pluralisation), `vite build` clean, HA config check and startup clean.
+
+**Still open / worth knowing:** the import has only ever run against seeded data, never against a
+real multi-week recorder history on the production instance — the first real run is the honest test,
+especially for how long a full year takes (the UI waits up to two minutes before saying it continues
+in the background). Address reverse-geocoding is not attempted for imported trips (`start_address`/
+`end_address` stay null, same as live-detected ones before the user edits them).
+
+### G) Fresh-install audit + installer hardening (2026-08-23, before the first real deployment)
+
+Owner is about to install on the real HA OS instance and asked for two things: that the one-click
+package produce a genuinely blank app, and that it carry no risk to the existing HA system.
+
+**The one real contamination bug.** `pyscript/modules/einstellungen.py` shipped with two hardcoded
+entity IDs from a long-gone test instance — `ZUENDUNG_SENSOR =
+"binary_sensor.testzone_fmm003_engine_ignition_or_acc_status"` and `BATTERIE_SENSOR =
+"sensor.testzone_fmm003_external_power_voltage"` (note `testzone_`, older even than the current test
+container's `testcar_b9_…`). Invisible during all testing, because the test instance's own
+`entitaeten.json` overrides both. On a fresh install with no overrides they would have been the
+*effective* values — and worse than merely dangling: `fahrterkennung.py` guards trigger registration
+with `if einstellungen.ZUENDUNG_SENSOR:`, explicitly built so an unset sensor registers no trigger at
+all. A set-but-nonexistent value defeats exactly that guard. Both are now `""` like every other field,
+and the file header was rewritten — it still claimed the EU-Data-Act integration was gone, which
+stopped being true when `cupra_eu_data_act` appeared.
+
+Everything else in the package was already clean, verified rather than assumed: no `fahrten.jsonl`,
+no `tankvorgaenge.jsonl`, no `batteriespannung.jsonl`, no `entitaeten.json`, no real tokens (the
+`ha_token` hits are path constants), and `fahrzeugprofil.example.json` carries only model specs —
+FIN, Kennzeichen, Erstzulassung, HU, insurance numbers and all counters are empty or zero.
+
+**Verified by actually doing it**, not by reading the installer: a throwaway `ghcr.io/home-assistant/
+home-assistant:stable` container, the package copied in exactly as `install.ps1` does it, pyscript
+config entry added, restarted. Result: `/config/audi_dashboard/` held only `fahrzeugprofil.json`,
+`shell_beleg_parser.py` and an empty `belege/`; no trip, fuel or battery file existed at all; zero
+ERROR lines in the log.
+
+**Installer safety, audited and then hardened.** The audit found the good news first: `install.ps1`
+contains no `Remove-Item` anywhere, no `robocopy /MIR`, never touches `.storage/`, never reads or
+writes `automations.yaml`/`scripts.yaml`/`secrets.yaml`, and never restarts HA. The only file it
+changes outside its own folders is `configuration.yaml` — which is also the only way it could stop HA
+booting. Three gaps closed there:
+
+- the fixed `configuration.yaml.bak` was overwritten on every run, so a second run destroyed the
+  pristine original. Backups are now timestamped, and the plain `.bak` is written only once
+- after writing, the file is read back and checked (still at least as long as before, marker block
+  present exactly once, prior content byte-identical); any deviation **rolls back automatically**
+  from the timestamped backup and moves the step to the manual to-do list
+- the header now states the safety properties explicitly, so they can be checked rather than trusted
+
+Tested end to end against mock config dirs: `-Pruefen` wrote nothing; a real run preserved a foreign
+`light:` group and left `automations.yaml` byte-identical (SHA-256 compared) while appending the
+block; the output parses as valid HA YAML with all foreign keys intact; a second run left exactly one
+block and preserved `fahrzeugprofil.json`; and against a config that already had its own `pyscript:`
+and `panel_custom:` the file came back byte-identical, as designed.
+
+**The genuine remaining risk is disk space, not the installer.** `recorder_snippet.yaml`'s 365 days
+is ~1–1,5 GB, and on HA OS with an SD card or small eMMC a full disk stops HA from starting — nothing
+to do with this app. The snippet header and the installer's to-do list now say so plainly, tell the
+user to check Einstellungen → System → Speicher first, and recommend `purge_keep_days: 90` when space
+is tight (enough for the "catch up on the past" purpose, since the import moves data into the app's
+own permanent `.jsonl` files anyway).
+
+**Also fixed this round (panel-only):** the "Teilen" pill on the location sheet was invisible in
+day mode — `.standort-pille` used `background: var(--tile)` while the sheet under it uses
+`var(--tile-deckend)`, and the iOS overlay sets *both* to `#FFFFFF` in day mode. Measured contrast
+was 1.00:1, white on white. Both pills now follow the app's own button language (`.aktion` /
+`.aktion.primaer`): outline for the secondary, `var(--fg)` fill for "Route". Text contrast measured
+17–21:1 in both themes afterwards. This also retired `--line-strong` — a *line* colour — being used
+as a button fill. No companion-app port: the location sheet's share/route pills do not exist there,
+which the parity rule exempts as panel-only.
+
 ---
 
 ## Working conventions (observed — keep them)