{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "2569e8aa",
   "metadata": {},
   "source": [
    "# 09 · The CLI and serializable jobs\n",
    "\n",
    "Everything so far has been Python. For **remote and batch** compute you want a\n",
    "job you can hand to another machine — a cluster node, a cloud runner — without\n",
    "shipping your session. `gds_fdtd` gives you two things for that:\n",
    "\n",
    "- a **`JobSpec`**: one JSON document that fully describes a simulation\n",
    "  (layout file, technology file, `SimulationSpec`, engine, budget) — no live\n",
    "  objects, no secrets;\n",
    "- the **`gds-fdtd` CLI**: `solvers | validate | build | estimate | run |\n",
    "  convert | convert-tech`, with an explicit exit-code contract.\n",
    "\n",
    "`validate` / `build` / `estimate` are **offline and free** — the CLI previews a\n",
    "job (and its cost) before `run` ever touches an engine. This whole notebook\n",
    "runs offline."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "d6972cba",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-07-11T03:45:59.595290Z",
     "iopub.status.busy": "2026-07-11T03:45:59.595128Z",
     "iopub.status.idle": "2026-07-11T03:45:59.760688Z",
     "shell.execute_reply": "2026-07-11T03:45:59.760401Z"
    }
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import subprocess\n",
    "import sys\n",
    "import tempfile\n",
    "from pathlib import Path\n",
    "\n",
    "from gds_fdtd.execution import JobSpec\n",
    "\n",
    "# Run from the repo root so every path in the job is relative and portable.\n",
    "_root = next(b for b in (Path.cwd(), *Path.cwd().parents) if (b / \"pyproject.toml\").exists())\n",
    "os.chdir(_root)\n",
    "\n",
    "\n",
    "def _cli(*args: str) -> None:\n",
    "    \"\"\"Run `python -m gds_fdtd.cli ...` and show the exit code + output.\"\"\"\n",
    "    r = subprocess.run(\n",
    "        [sys.executable, \"-m\", \"gds_fdtd.cli\", *args], capture_output=True, text=True\n",
    "    )\n",
    "    print(f\"$ gds-fdtd {' '.join(args)}   →  exit {r.returncode}\")\n",
    "    print((r.stdout or r.stderr).rstrip() or \"(no output)\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "00fd32ee",
   "metadata": {},
   "source": [
    "## 1 · A job as a file\n",
    "\n",
    "A `JobSpec` references its layout and technology **by path** and carries the\n",
    "validated `SimulationSpec`. We point it at the Si→SiN escalator layout from\n",
    "`10_cookbook` and serialize the whole job to JSON."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "abda56e6",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-07-11T03:45:59.761920Z",
     "iopub.status.busy": "2026-07-11T03:45:59.761850Z",
     "iopub.status.idle": "2026-07-11T03:45:59.764663Z",
     "shell.execute_reply": "2026-07-11T03:45:59.764337Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"gds_path\": \"examples/10_cookbook/si_sin_escalator.gds\",\n",
      "  \"top_cell\": null,\n",
      "  \"technology_path\": \"examples/tech.yaml\",\n",
      "  \"spec\": {\n",
      "    \"wavelength_start\": 1.5,\n",
      "    \"wavelength_end\": 1.6,\n",
      "    \"wavelength_points\": 5,\n",
      "    \"mesh\": 6,\n",
      "    \"boundary\": [\n",
      "      \"PML\",\n",
      "      \"PML\",\n",
      "      \"PML\"\n",
      "    ],\n",
      "    \"symmetry\": [\n",
      "      0,\n",
      "      0,\n",
      "      0\n",
      "    ],\n",
      "    \"z_min\": -1.0,\n",
      "    \"z_max\": 1.11,\n",
      "    \"width_ports\": 2.0,\n",
      "    \"depth_ports\": 1.5,\n",
      "    \"buffer\": 1.0,\n",
      "    \"modes\": [\n",
      "      1\n",
      "    ],\n",
      "    \"mode_freq_pts\": 3,\n",
      "    \"run_time_factor\": 3.0,\n",
      "    \"field_monitors\": [\n",
      "      \"z\"\n",
      "    ]\n",
      "  },\n",
      "  \"solver\": \"beamz\",\n",
      "  \"solver_options\": {},\n",
      "  \"budget\": null\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "workdir = Path(tempfile.mkdtemp(prefix=\"gdsfdtd_job_\"))\n",
    "job = JobSpec(\n",
    "    gds_path=\"examples/10_cookbook/si_sin_escalator.gds\",\n",
    "    technology_path=\"examples/tech.yaml\",\n",
    "    solver=\"beamz\",\n",
    "    spec={\"wavelength_start\": 1.5, \"wavelength_end\": 1.6, \"wavelength_points\": 5, \"mesh\": 6,\n",
    "          \"z_min\": -1.0, \"z_max\": 1.11},\n",
    ")\n",
    "job.to_file(workdir / \"job.json\")\n",
    "print((workdir / \"job.json\").read_text())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dd461e56",
   "metadata": {},
   "source": [
    "The job is pure JSON — safe to commit, ship, or queue. Secrets (API keys,\n",
    "license servers) stay in the environment of whatever machine runs it, never in\n",
    "the file. It round-trips exactly:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "e2f00138",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-07-11T03:45:59.765632Z",
     "iopub.status.busy": "2026-07-11T03:45:59.765588Z",
     "iopub.status.idle": "2026-07-11T03:45:59.767466Z",
     "shell.execute_reply": "2026-07-11T03:45:59.767207Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "round-trips identically: True\n"
     ]
    }
   ],
   "source": [
    "reloaded = JobSpec.from_file(workdir / \"job.json\")\n",
    "print(\"round-trips identically:\", reloaded == job)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8e1c418a",
   "metadata": {},
   "source": [
    "## 2 · Driving it from the shell\n",
    "\n",
    "`solvers` reports the engines registered on this machine and their cost model —\n",
    "handy before you pick one for a job:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "5795d88f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-07-11T03:45:59.768190Z",
     "iopub.status.busy": "2026-07-11T03:45:59.768149Z",
     "iopub.status.idle": "2026-07-11T03:46:01.645274Z",
     "shell.execute_reply": "2026-07-11T03:46:01.644771Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "$ gds-fdtd solvers   →  exit 0\n",
      "beamz        ok                   execution=local cost=free multimode=False\n",
      "lumerical    ok                   execution=local cost=licensed multimode=True\n",
      "tidy3d       ok                   execution=cloud cost=credits multimode=True\n"
     ]
    }
   ],
   "source": [
    "_cli(\"solvers\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d5edd6db",
   "metadata": {},
   "source": [
    "`validate` / `estimate` / `build` preview the job **offline** (exit 0 = ready;\n",
    "2 = the job has problems; 3 = the engine isn't usable here; 4 = over budget):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "0c1bfac0",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-07-11T03:46:01.646356Z",
     "iopub.status.busy": "2026-07-11T03:46:01.646296Z",
     "iopub.status.idle": "2026-07-11T03:46:07.004785Z",
     "shell.execute_reply": "2026-07-11T03:46:07.004459Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "$ gds-fdtd validate /var/folders/44/xc__wmg576n7crzyny00z3j80000gp/T/gdsfdtd_job_7q2yai78/job.json   →  exit 0\n",
      "valid: True\n",
      "problems: []\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "$ gds-fdtd estimate /var/folders/44/xc__wmg576n7crzyny00z3j80000gp/T/gdsfdtd_job_7q2yai78/job.json   →  exit 0\n",
      "● Done: Raster cache hit (3d): 564a94fc471723166acc7b252b3f32472ee51d17081d05fd1a597a4b1c6cfbed.npz | load=0.04s\n",
      "grid_cells: 1960980\n",
      "memory_gb: 0.09412704\n",
      "n_simulations: 2\n",
      "cost_hint: free local compute (JAX; CPU works, GPU if available)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "$ gds-fdtd build /var/folders/44/xc__wmg576n7crzyny00z3j80000gp/T/gdsfdtd_job_7q2yai78/job.json   →  exit 0\n",
      "● Done: Raster cache hit (3d): 564a94fc471723166acc7b252b3f32472ee51d17081d05fd1a597a4b1c6cfbed.npz | load=0.04s\n",
      "files: {}\n",
      "summary: {'n_ports': 2, 'grid_shape': (70, 87, 322), 'dx_nm': 74.31914077483697, 'n_core': 3.476, 'n_clad': 1.444, 'n_simulations': 2}\n"
     ]
    }
   ],
   "source": [
    "_cli(\"validate\", str(workdir / \"job.json\"))\n",
    "_cli(\"estimate\", str(workdir / \"job.json\"))\n",
    "_cli(\"build\", str(workdir / \"job.json\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9cb666db",
   "metadata": {},
   "source": [
    "## 3 · Running — locally or crash-isolated\n",
    "\n",
    "`run` executes the job; `--backend subprocess` runs it in a **separate process**\n",
    "(so one bad job can't take down a sweep) and enforces the job's wall-clock\n",
    "budget. The subprocess backend re-invokes this same CLI, which is why\n",
    "the JSON boundary matters — the child rebuilds the job from the file alone:\n",
    "\n",
    "```bash\n",
    "gds-fdtd run job.json --out results/                 # in-process\n",
    "gds-fdtd run job.json --out results/ --backend subprocess   # crash-isolated\n",
    "```\n",
    "\n",
    "```python\n",
    "from gds_fdtd.execution import SubprocessBackend\n",
    "handle = SubprocessBackend().submit(job, out_dir=\"results\")\n",
    "result = handle.result()          # JobResult: smatrix_path, job_hash, wall_seconds, ...\n",
    "```\n",
    "\n",
    "`run` is the only verb that spends — and a `Budget` on the `JobSpec`\n",
    "(`max_flexcredits`, `max_wall_seconds`) lets the CLI **refuse before spending**\n",
    "(exit 4). See `docs/remote_compute.md` for Modal / AWS / SLURM recipes.\n",
    "\n",
    "## Recap & next\n",
    "\n",
    "A simulation is a portable JSON `JobSpec`; the `gds-fdtd` CLI previews it for\n",
    "free and runs it locally or on a cluster, with budgets enforced up front. Next:\n",
    "**`10_cookbook`** — reference devices with known-good S-parameters."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}