FiniteHorizonMDPDP—Topologies&DiscountTools℠🅪™©💡

Finite-Horizon Optimal Control (Markov DP) with topology choices and discount calculators.

FHOC MDP DP — Realistic v6℠🅪™©💡

• Die institutionelle, Enterprise Edition, Corporate Structured.
• The Institutional‑Grade, B2B, D.A.O.Enterprise Edition, Corporate Consumers Structured Add-on.

Contains 5 lightweight and 2 entire versions in it.

FHOC MDP DP — Realistic v9 (Horizon estimator added; robust auto-estimate flow)

This release fixes the missing horizon estimator and tightens the auto-estimate flow:

- Adds estimate_horizon_from_p_move_and_target function (previously missing).

- Ensures horizon auto-estimate runs before DP solve and uses the final p_move estimate.

- Keeps prior robustness: deterministic topology builders, terminal-value-aware DP,

 adaptive tie-break, small state-dependent bias to break symmetry, threaded GUI,

 discrete policy heatmap, CSV/manifest export.

- Validates and clamps inputs to avoid pathological values.

Features:

- Three topologies: linear chain, radial hub-and-spoke, global sparse graph, extendable

- Two actions preserved: WAIT (stay) and MOVE (attempt transition along edges)

- Discount factor computed from annual rate, half-life, or custom gamma

- Backward induction DP using general transition matrices P_wait and P_move

- Tkinter GUI to enter parameters, compute gamma, build topology, solve DP, plot results

- Plots: value function curves and policy heatmap

Finite-Horizon MDP DP with deterministic topology building and robust tie-breaking.

- Three topologies: linear, radial, global_sparse

- Deterministic RNG seed input for reproducibility

- Tolerance-based tie-break: choose MOVE only if Q_move > Q_wait + tol

- Gamma calculators (annual rate, half-life, custom)

- Tkinter GUI, plotting and policy display

Finite-Horizon MDP DP — Deterministic, robust, Earth-aware enhancements.

Key features added:

- Deterministic topology construction (user-provided RNG seed)

- Robust tie-breaking with tolerance to avoid numerical flip-flops

- Multiple realistic discount modes:

  * annual rate (nominal)

  * half-life

  * continuous discount (exp(-r*dt))

  * risk-free rate

  * social discount rate

  * climate-aware discount (Stern-like / Nordhaus-like presets)

- Automatic terminal-reward estimation helper:

  * If user supplies node metadata (population, GDP_per_capita, importance score),

   the app computes terminal reward from a weighted combination.

  * If no metadata provided, the app uses topology-based heuristics (hub centrality,

   degree, or distance-to-hub) to assign terminal rewards.

  * All estimators are deterministic and reproducible (seeded).

- Deterministic greedy simulation (seeded)

- Exportable run manifest (parameters + seed) for reproducibility

- Tkinter GUI for parameter entry, gamma computation, topology build, DP solve, plotting

- CSV export of policy/value tables (optional)

- Logging of run manifest to outputs/run_manifest.json

- For production Earth-scale routing, replace the synthetic topology builders with a domain-specific graph (e.g., nodes = geo-locations,edges = routes with travel times and probabilities) and calibrate rewards/costs.

FHOC MDP DP — Deterministic, Robust, Earth-aware v2

Improvements:

- Deterministic topology RNG seed

- Tolerance-based tie-break to avoid numerical flip-flops

- Auto-estimation of p_move, cost_wait, cost_move, terminal_rewards, and horizon H

 using topology heuristics and optional node metadata

- Multiple realistic discount modes

- Export run manifest and CSV of V/policy

- GUI (tkinter) for interactive use

FHOC MDP DP — Realistic v3 (fixed hangs, deterministic, robust, auto-estimation)

- Fixes Build Topology hang by using vectorized neighbor sampling (no while loops)

- Runs expensive tasks (build topology, solve DP) in background threads to keep GUI responsive

- Validates inputs and caps parameters to safe ranges to avoid extreme terminal rewards

- Ensures P_move uses the estimated p_move consistently

- Adds deterministic RNG seeding and reproducible outputs

- Exports run manifest and CSV; saves outputs to ./outputs/

Requires: Python 3.8+, numpy, matplotlib, tkinter

FHOC MDP DP — Realistic v6

# Horizon estimator (ADDED)

Key fixes and improvements over prior versions:

- Correct, robust gamma computation (reads the selected discount mode and inputs)

- Ensures P_move is rebuilt consistently from the final estimated p_move

- Fixes seeded greedy trajectory producing all zeros:

  * policy values are 0/1 and indexed correctly

  * simulation uses the policy action for the current state (no off-by-one)

  * adds small deterministic epsilon-exploration option so trajectories show movement

   when the policy is near-indifferent (configurable)

- Avoids GUI hangs by running heavy tasks in background threads

- Vectorized, deterministic topology builders (no infinite loops)

- Diagnostics and adaptive tie-break: if policy is degenerate (all WAIT) but Q differences

 are near zero, recompute with relaxed tie-break to reflect real-world indifference

- Input validation and clamping to avoid pathological values

- Exports run manifest and CSV to ./outputs/

Contact the author for pass-codes to activate products upon purchase.

- Clean, production-ready, reproducible defaults and deterministic RNG seeding