notebooks/schema_mining_with_LLMs_and_expert_example.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 📄🔍 A Human-in-the-Loop Workflow for Scientific Schema Mining with Large Language Models"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook presents a human-in-the-loop approach for scientific schema mining utilizing large language models (LLMs). It outlines the steps for creating and preparing the data necessary for schema mining, and describes the three stages involved in extracting and mining JSON schemas from scientific literature, incorporating continuous expert feedback. A complete workflow diagram and detailed explanation can be found [here](../README.md).\n",
    "\n",
    "In this notebook, we will be taking the example of an ALD (Atomic Layer Deposition) process and design the schema for it!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Overview**\n",
    "1. [Preparing the data](#preparing-data)\n",
    "2. [Stage 01: Initial Schema Mining](#initial-schema-mining)\n",
    "3. [Stage 02: Preliminary Schema Refinement](#pre-schema-refinement)\n",
    "4. [Stage 03: Finalize Schema Refinement](#finalize-schema-refinement)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 🗂️ Preparing the data <a id='preparing-data'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The initial step before schema mining is to prepare the data for the large language model (LLM). This preparation involves reviewing various PDF documents related to Atomic Layer Deposition (ALD) processes across different stages. For Stage 1, we will use the process specification document provided by domain experts to extract the initial JSON schema. For Stages 2 and 3, we will utilize a curated dataset, compiled by domain experts, containing various scientific research papers on the ALD process. All relevant documents are organized and available [here](../data)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "from utils.utils import pdf_loader\n",
    "\n",
    "def convert_pdf_to_text(source_dir_path: str):\n",
    "    \"\"\"\n",
    "    Extracts text from all the PDF's in the source directory\n",
    "\n",
    "    Args:\n",
    "        source_dir_path (str): The file path of the source directory\n",
    "\n",
    "    Returns:\n",
    "        list: The list of string for each of the files in the source directory\n",
    "    \"\"\"\n",
    "    pdf_text_lst = []\n",
    "    \n",
    "    #Iterating over all the files in the source directory\n",
    "    for filename in os.listdir(source_dir_path):\n",
    "        #Checking for the PDF file\n",
    "        if not filename.endswith('.pdf'): continue\n",
    "\n",
    "        #Reading and extracting the pdf text\n",
    "        complete_file_path = f'{source_dir_path}/{filename}'\n",
    "        pdf_text = pdf_loader(complete_file_path)\n",
    "        pdf_text_lst.append(pdf_text)\n",
    "    return pdf_text_lst"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Reading the process specification documents for stage 1\n",
    "source_dir_path = '../data/stage-1'\n",
    "stage_1_text = convert_pdf_to_text(source_dir_path)\n",
    "\n",
    "#Reading the domain-expert's small curated collection of research papers for stage 2\n",
    "source_dir_path = '../data/stage-2/research-papers/experimental-usecase'\n",
    "stage_2_text = convert_pdf_to_text(source_dir_path)\n",
    "\n",
    "#Reading the large collection of research papers for stage 3\n",
    "source_dir_path = '../data/stage-3/research-papers/experimental_usecase'\n",
    "stage_3_text = convert_pdf_to_text(source_dir_path)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 📄⛏️ Stage 01: Initial Schema Mining <a id='initial-schema-mining'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "During Stage 1, we task the LLM with designing an initial JSON schema based on a process specification document that contains limited information about an ALD process. For this example, we have used OpenAI's GPT-4 language model, using a prompt template as demonstrated [here](../src/prompts/prompt_template1.py)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "from utils.utils import extract_json_schema\n",
    "from services.openai_llm_inference import Openai_LLM_Inference\n",
    "from prompts import prompt_template1\n",
    "\n",
    "def schema_extraction_stage1(llm_model_name: str, specification_doc: str):\n",
    "    \"\"\"\n",
    "    Extract the Initial JSON schema from the given process specification document and the large language model.\n",
    "\n",
    "    Args:\n",
    "        llm_model_name (str): The large language model name\n",
    "        specification_doc (str): The text of the process specification document\n",
    "\n",
    "    Returns:\n",
    "        dict: The extracted Initial Schema\n",
    "    \"\"\"\n",
    "    try:\n",
    "        #Initializing the Object for LLM inference\n",
    "        llm = Openai_LLM_Inference(llm_model_name)\n",
    "        print(f'Using {llm}')\n",
    "        \n",
    "        #Calling the LLM's completion API\n",
    "        var_dict = {'context': specification_doc}\n",
    "        model_output = llm.completion(prompt_template1, var_dict)\n",
    "        if not model_output:\n",
    "            print(f'No Output from the Model: {llm_model_name}, terminating the workflow')\n",
    "            return None\n",
    "\n",
    "        #Extracting the updated schema\n",
    "        print('\\nExtracting the JSON object from the model\\'s output...')\n",
    "        initial_schema = extract_json_schema(model_output, json_encl_expr = [('```json', '```'), ('```', '```')])\n",
    "        if not initial_schema:\n",
    "            print('Error extracting the JSON schema from the LLM\\'s output, Stopping the inference from the LLM...')\n",
    "            return None\n",
    "        \n",
    "        print('\\nInitial Schema successfully extracted from the LLM\\'s Output!!')\n",
    "        return initial_schema\n",
    "    except Exception as e:\n",
    "            print(f'Exception Occurred while doing LLM ({llm_model_name}) Inference')\n",
    "            print(f'Exception: {e}')\n",
    "            return None"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using OPENAI - LLM Inference with Model: gpt-4o\n",
      "\n",
      "Extracting the JSON object from the model's output...\n",
      "\n",
      "Initial Schema successfully extracted from the LLM's Output!!\n"
     ]
    }
   ],
   "source": [
    "process_specification_doc = stage_1_text[0]\n",
    "llm_model_name = 'gpt-4o'\n",
    "initial_schema = schema_extraction_stage1(llm_model_name, process_specification_doc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Initial Schema:\n",
      "\n",
      "{\n",
      "    \"$schema\": \"http://json-schema.org/draft-07/schema#\",\n",
      "    \"title\": \"Atomic Layer Deposition (ALD) Process Schema\",\n",
      "    \"type\": \"object\",\n",
      "    \"properties\": {\n",
      "        \"processParameters\": {\n",
      "            \"type\": \"object\",\n",
      "            \"description\": \"Parameters defining the ALD process.\",\n",
      "            \"properties\": {\n",
      "                \"temperature\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Temperature at which the ALD process is conducted, in Celsius.\",\n",
      "                    \"units\": \"Celsius\"\n",
      "                },\n",
      "                \"pressure\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Pressure inside the reactor during the ALD process, in Pascals.\",\n",
      "                    \"units\": \"Pascals\"\n",
      "                },\n",
      "                \"precursorFlowRate\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Flow rate of the precursor gas, in standard cubic centimeters per minute (sccm).\",\n",
      "                    \"units\": \"sccm\"\n",
      "                },\n",
      "                \"purgeFlowRate\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Flow rate of the purge gas, in standard cubic centimeters per minute (sccm).\",\n",
      "                    \"units\": \"sccm\"\n",
      "                },\n",
      "                \"substrateVelocity\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Velocity of the substrate during the ALD process, in meters per second.\",\n",
      "                    \"units\": \"m/s\"\n",
      "                },\n",
      "                \"gapDistance\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Distance between the substrate and the precursor source, in millimeters.\",\n",
      "                    \"units\": \"mm\"\n",
      "                }\n",
      "            },\n",
      "            \"required\": [\n",
      "                \"temperature\",\n",
      "                \"pressure\",\n",
      "                \"precursorFlowRate\",\n",
      "                \"purgeFlowRate\"\n",
      "            ]\n",
      "        },\n",
      "        \"materialProperties\": {\n",
      "            \"type\": \"object\",\n",
      "            \"description\": \"Properties of the material being deposited.\",\n",
      "            \"properties\": {\n",
      "                \"growthRate\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Growth rate of the film, in nanometers per cycle.\",\n",
      "                    \"units\": \"nm/cycle\"\n",
      "                },\n",
      "                \"uniformity\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Uniformity of the film thickness across the substrate, as a percentage.\",\n",
      "                    \"units\": \"%\"\n",
      "                },\n",
      "                \"roughness\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Surface roughness of the deposited film, in nanometers.\",\n",
      "                    \"units\": \"nm\"\n",
      "                },\n",
      "                \"density\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Density of the deposited film, in grams per cubic centimeter.\",\n",
      "                    \"units\": \"g/cm^3\"\n",
      "                },\n",
      "                \"chemicalComposition\": {\n",
      "                    \"type\": \"string\",\n",
      "                    \"description\": \"Chemical composition of the deposited film.\"\n",
      "                }\n",
      "            },\n",
      "            \"required\": [\n",
      "                \"growthRate\",\n",
      "                \"uniformity\",\n",
      "                \"roughness\",\n",
      "                \"density\"\n",
      "            ]\n",
      "        },\n",
      "        \"reactionMechanisms\": {\n",
      "            \"type\": \"object\",\n",
      "            \"description\": \"Details of the reaction mechanisms involved in the ALD process.\",\n",
      "            \"properties\": {\n",
      "                \"chemisorptionMechanism\": {\n",
      "                    \"type\": \"string\",\n",
      "                    \"description\": \"Mechanism of chemisorption during the ALD process.\"\n",
      "                },\n",
      "                \"bindingAffinity\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Binding affinity of the adsorbate, in kilojoules per mole.\",\n",
      "                    \"units\": \"kJ/mol\"\n",
      "                },\n",
      "                \"stickingCoefficient\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Sticking coefficient of the adsorbate.\"\n",
      "                },\n",
      "                \"surfaceCoverage\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Surface coverage by the adsorbate, as a percentage.\",\n",
      "                    \"units\": \"%\"\n",
      "                }\n",
      "            },\n",
      "            \"required\": [\n",
      "                \"chemisorptionMechanism\",\n",
      "                \"bindingAffinity\",\n",
      "                \"stickingCoefficient\"\n",
      "            ]\n",
      "        },\n",
      "        \"simulationMethods\": {\n",
      "            \"type\": \"array\",\n",
      "            \"description\": \"Simulation methods used to model the ALD process.\",\n",
      "            \"items\": {\n",
      "                \"type\": \"string\",\n",
      "                \"enum\": [\n",
      "                    \"DFT\",\n",
      "                    \"kMC\",\n",
      "                    \"MD\",\n",
      "                    \"CFD\",\n",
      "                    \"LBM\",\n",
      "                    \"GCM\",\n",
      "                    \"CAMD\",\n",
      "                    \"HF\",\n",
      "                    \"ML\"\n",
      "                ]\n",
      "            }\n",
      "        }\n",
      "    },\n",
      "    \"required\": [\n",
      "        \"processParameters\",\n",
      "        \"materialProperties\",\n",
      "        \"reactionMechanisms\",\n",
      "        \"simulationMethods\"\n",
      "    ]\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "print('Initial Schema:\\n')\n",
    "print(json.dumps(initial_schema, indent=4))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "expert_feedback_stage1 = 'The reactivity should be mentioned either at the process conditions or film properties. It’s the result of deposition under specific conditions not a standard property of the precursor or co-reactant.'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 📄⛏️ Stage 02: Preliminary Schema Refinement <a id='pre-schema-refinement'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "During Stage 2, the initial JSON schema from Stage 1 is further refined using a curated dataset of scientific literature related to the ALD process, compiled by domain experts. This stage follows an iterative approach, incorporating expert feedback into each schema generated by the LLM. Here in this notebook, we have used only one expert's feedback to demostrate how it can be incorporated into the workflow. The prompt used for this stage is provided [here](../src/prompts/prompt_template2.py)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "from prompts import prompt_template2\n",
    "\n",
    "def schema_extraction_stage2(llm_model_name: str, schema: dict, review: str, scientific_literature: list):\n",
    "    \"\"\"\n",
    "    Updates the initial JSON schema with the small domain-expert's curated scientific literature and continous expert's feedback\n",
    "\n",
    "    Args:\n",
    "        llm_model_name (str): The large language model name\n",
    "        schema (dict): The initial schema from stage 1\n",
    "        review (str): The expert's feedback on the initial schema from stage 1\n",
    "        scientific_literature (list): The list of text of domain-expert's curated scientific literature\n",
    "\n",
    "    Returns:\n",
    "        dict: The updated JSON schema\n",
    "    \"\"\"\n",
    "    \n",
    "    #Initializing the object for LLM Inference\n",
    "    llm = Openai_LLM_Inference(llm_model_name)\n",
    "    print(f'Using {llm}')\n",
    "    \n",
    "    #Initializing the counter\n",
    "    index = 0\n",
    "    \n",
    "    #Iterating over the scientific literature\n",
    "    for sci_lit in scientific_literature:\n",
    "        print(f'\\n========================== Iteration {index + 1} =================================')\n",
    "        \n",
    "        #Extracting the schema from the LLM\n",
    "        print('\\nCalling the completion API of the model')\n",
    "        var_dict = {'current_schema': schema, 'full_text': sci_lit, 'domain_expert_review': review}\n",
    "        model_output = llm.completion(prompt_template2, var_dict)\n",
    "        \n",
    "        #Returns None, if any exception occurrs during the LLM Inference\n",
    "        if not model_output:\n",
    "            print(f'No output from model: {llm_model_name}, continuing to the next paper..')\n",
    "            continue\n",
    "        \n",
    "        #Extracting the updated schema\n",
    "        updated_schema = extract_json_schema(model_output, json_encl_expr = [('```json', '```'), ('```', '```')])\n",
    "        if not updated_schema:\n",
    "            print('Unable to extract JSON, stopping the inference from the LLM...')\n",
    "            break\n",
    "        \n",
    "        #Updating the previous schema with the current schema for next iteration\n",
    "        schema = updated_schema\n",
    "        index += 1\n",
    "        \n",
    "    print(f'JSON schema updated with LLM: {llm_model_name}')\n",
    "    return schema"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using OPENAI - LLM Inference with Model: gpt-4o\n",
      "\n",
      "========================== Iteration 1 =================================\n",
      "\n",
      "Calling the completion API of the model\n",
      "\n",
      "========================== Iteration 2 =================================\n",
      "\n",
      "Calling the completion API of the model\n",
      "\n",
      "========================== Iteration 3 =================================\n",
      "\n",
      "Calling the completion API of the model\n",
      "\n",
      "========================== Iteration 4 =================================\n",
      "\n",
      "Calling the completion API of the model\n",
      "\n",
      "========================== Iteration 5 =================================\n",
      "\n",
      "Calling the completion API of the model\n",
      "JSON schema updated with LLM: gpt-4o\n"
     ]
    }
   ],
   "source": [
    "llm_model_name = 'gpt-4o'\n",
    "stage2_schema = schema_extraction_stage2(llm_model_name, initial_schema, expert_feedback_stage1, stage_2_text[:5])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stage-2 Updated Schema:\n",
      "\n",
      "{\n",
      "    \"$schema\": \"http://json-schema.org/draft-07/schema#\",\n",
      "    \"title\": \"Atomic Layer Deposition (ALD) Process Schema\",\n",
      "    \"type\": \"object\",\n",
      "    \"properties\": {\n",
      "        \"processParameters\": {\n",
      "            \"type\": \"object\",\n",
      "            \"description\": \"Parameters defining the ALD process.\",\n",
      "            \"properties\": {\n",
      "                \"temperature\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Temperature at which the ALD process is conducted, in Celsius. Typical range is 50 to 500 \\u00b0C.\",\n",
      "                    \"units\": \"Celsius\"\n",
      "                },\n",
      "                \"pressure\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Pressure inside the reactor during the ALD process, typically maintained at 0.25 Torr.\",\n",
      "                    \"units\": \"Pascals\"\n",
      "                },\n",
      "                \"precursorFlowRate\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Flow rate of the precursor gas, in standard cubic centimeters per minute (sccm).\",\n",
      "                    \"units\": \"sccm\"\n",
      "                },\n",
      "                \"purgeFlowRate\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Flow rate of the purge gas, in standard cubic centimeters per minute (sccm).\",\n",
      "                    \"units\": \"sccm\"\n",
      "                },\n",
      "                \"substrateVelocity\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Velocity of the substrate during the ALD process, in meters per second.\",\n",
      "                    \"units\": \"m/s\"\n",
      "                },\n",
      "                \"gapDistance\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Distance between the substrate and the precursor source, in millimeters.\",\n",
      "                    \"units\": \"mm\"\n",
      "                },\n",
      "                \"precursorType\": {\n",
      "                    \"type\": \"string\",\n",
      "                    \"description\": \"Type of precursor used in the ALD process, e.g., metal amides.\"\n",
      "                },\n",
      "                \"precursorPulseTime\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Duration of the precursor pulse, in seconds.\",\n",
      "                    \"units\": \"seconds\"\n",
      "                },\n",
      "                \"coReactantPulseTime\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Duration of the co-reactant pulse, in seconds.\",\n",
      "                    \"units\": \"seconds\"\n",
      "                },\n",
      "                \"vaporPressure\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Vapor pressure of the precursor, in Pascals.\",\n",
      "                    \"units\": \"Pascals\"\n",
      "                },\n",
      "                \"reactivity\": {\n",
      "                    \"type\": \"string\",\n",
      "                    \"description\": \"Reactivity of the precursors with hydroxylated surfaces, influencing deposition efficiency.\"\n",
      "                }\n",
      "            },\n",
      "            \"required\": [\n",
      "                \"temperature\",\n",
      "                \"pressure\",\n",
      "                \"precursorFlowRate\",\n",
      "                \"purgeFlowRate\"\n",
      "            ]\n",
      "        },\n",
      "        \"materialProperties\": {\n",
      "            \"type\": \"object\",\n",
      "            \"description\": \"Properties of the material being deposited.\",\n",
      "            \"properties\": {\n",
      "                \"growthRate\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Growth rate of the film, in nanometers per cycle.\",\n",
      "                    \"units\": \"nm/cycle\"\n",
      "                },\n",
      "                \"uniformity\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Uniformity of the film thickness across the substrate, as a percentage.\",\n",
      "                    \"units\": \"%\"\n",
      "                },\n",
      "                \"roughness\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Surface roughness of the deposited film, in nanometers.\",\n",
      "                    \"units\": \"nm\"\n",
      "                },\n",
      "                \"density\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Density of the deposited film, in grams per cubic centimeter.\",\n",
      "                    \"units\": \"g/cm^3\"\n",
      "                },\n",
      "                \"chemicalComposition\": {\n",
      "                    \"type\": \"string\",\n",
      "                    \"description\": \"Chemical composition of the deposited film.\"\n",
      "                },\n",
      "                \"electricalProperties\": {\n",
      "                    \"type\": \"object\",\n",
      "                    \"description\": \"Electrical properties of the deposited film.\",\n",
      "                    \"properties\": {\n",
      "                        \"leakageCurrentDensity\": {\n",
      "                            \"type\": \"number\",\n",
      "                            \"description\": \"Leakage current density of the film, in nanoamperes per square centimeter.\",\n",
      "                            \"units\": \"nA/cm^2\"\n",
      "                        },\n",
      "                        \"breakdownField\": {\n",
      "                            \"type\": \"number\",\n",
      "                            \"description\": \"Breakdown field of the film, in megavolts per centimeter.\",\n",
      "                            \"units\": \"MV/cm\"\n",
      "                        },\n",
      "                        \"dielectricConstant\": {\n",
      "                            \"type\": \"number\",\n",
      "                            \"description\": \"Dielectric constant of the film.\"\n",
      "                        }\n",
      "                    }\n",
      "                },\n",
      "                \"gasDiffusionProperties\": {\n",
      "                    \"type\": \"object\",\n",
      "                    \"description\": \"Gas diffusion properties of the film.\",\n",
      "                    \"properties\": {\n",
      "                        \"CO2LossRate\": {\n",
      "                            \"type\": \"number\",\n",
      "                            \"description\": \"CO2 loss rate through the film, in cubic centimeters per day per bar per square centimeter.\",\n",
      "                            \"units\": \"cm^3/(day bar cm^2)\"\n",
      "                        },\n",
      "                        \"barrierImprovementFactor\": {\n",
      "                            \"type\": \"number\",\n",
      "                            \"description\": \"Barrier improvement factor for gas diffusion.\"\n",
      "                        }\n",
      "                    }\n",
      "                },\n",
      "                \"conformality\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Degree of conformality of the film, as a percentage of step coverage.\",\n",
      "                    \"units\": \"%\"\n",
      "                },\n",
      "                \"filmOrientation\": {\n",
      "                    \"type\": \"string\",\n",
      "                    \"description\": \"Crystallographic orientation of the deposited film.\"\n",
      "                },\n",
      "                \"impurityContent\": {\n",
      "                    \"type\": \"object\",\n",
      "                    \"description\": \"Impurity content in the deposited film.\",\n",
      "                    \"properties\": {\n",
      "                        \"hydrogen\": {\n",
      "                            \"type\": \"number\",\n",
      "                            \"description\": \"Hydrogen content in atomic percent.\",\n",
      "                            \"units\": \"at.%\"\n",
      "                        },\n",
      "                        \"carbon\": {\n",
      "                            \"type\": \"number\",\n",
      "                            \"description\": \"Carbon content in atomic percent.\",\n",
      "                            \"units\": \"at.%\"\n",
      "                        },\n",
      "                        \"oxygen\": {\n",
      "                            \"type\": \"number\",\n",
      "                            \"description\": \"Oxygen content in atomic percent.\",\n",
      "                            \"units\": \"at.%\"\n",
      "                        }\n",
      "                    }\n",
      "                },\n",
      "                \"adhesion\": {\n",
      "                    \"type\": \"string\",\n",
      "                    \"description\": \"Adhesion quality of the film to the substrate.\"\n",
      "                },\n",
      "                \"refractiveIndex\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Refractive index of the film at a specified wavelength.\",\n",
      "                    \"units\": \"dimensionless\"\n",
      "                },\n",
      "                \"filmThickness\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Thickness of the deposited film, in nanometers.\",\n",
      "                    \"units\": \"nm\"\n",
      "                },\n",
      "                \"porosity\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Porosity of the film, as a percentage.\",\n",
      "                    \"units\": \"%\"\n",
      "                },\n",
      "                \"resistivity\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Resistivity of the film, in microohm-centimeters.\",\n",
      "                    \"units\": \"\\u00b5\\u03a9\\u00b7cm\"\n",
      "                }\n",
      "            },\n",
      "            \"required\": [\n",
      "                \"growthRate\",\n",
      "                \"uniformity\",\n",
      "                \"roughness\",\n",
      "                \"density\"\n",
      "            ]\n",
      "        },\n",
      "        \"reactionMechanisms\": {\n",
      "            \"type\": \"object\",\n",
      "            \"description\": \"Details of the reaction mechanisms involved in the ALD process.\",\n",
      "            \"properties\": {\n",
      "                \"chemisorptionMechanism\": {\n",
      "                    \"type\": \"string\",\n",
      "                    \"description\": \"Mechanism of chemisorption during the ALD process.\"\n",
      "                },\n",
      "                \"bindingAffinity\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Binding affinity of the adsorbate, in kilojoules per mole.\",\n",
      "                    \"units\": \"kJ/mol\"\n",
      "                },\n",
      "                \"stickingCoefficient\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Sticking coefficient of the adsorbate.\"\n",
      "                },\n",
      "                \"surfaceCoverage\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Surface coverage by the adsorbate, as a percentage.\",\n",
      "                    \"units\": \"%\"\n",
      "                }\n",
      "            },\n",
      "            \"required\": [\n",
      "                \"chemisorptionMechanism\",\n",
      "                \"bindingAffinity\",\n",
      "                \"stickingCoefficient\"\n",
      "            ]\n",
      "        },\n",
      "        \"simulationMethods\": {\n",
      "            \"type\": \"array\",\n",
      "            \"description\": \"Simulation methods used to model the ALD process.\",\n",
      "            \"items\": {\n",
      "                \"type\": \"string\",\n",
      "                \"enum\": [\n",
      "                    \"DFT\",\n",
      "                    \"kMC\",\n",
      "                    \"MD\",\n",
      "                    \"CFD\",\n",
      "                    \"LBM\",\n",
      "                    \"GCM\",\n",
      "                    \"CAMD\",\n",
      "                    \"HF\",\n",
      "                    \"ML\"\n",
      "                ]\n",
      "            }\n",
      "        },\n",
      "        \"characterizationTechniques\": {\n",
      "            \"type\": \"object\",\n",
      "            \"description\": \"Techniques used to characterize the ALD process and film properties.\",\n",
      "            \"properties\": {\n",
      "                \"FTIR\": {\n",
      "                    \"type\": \"boolean\",\n",
      "                    \"description\": \"Indicates if Fourier Transform Infrared Spectroscopy was used.\"\n",
      "                },\n",
      "                \"QCM\": {\n",
      "                    \"type\": \"boolean\",\n",
      "                    \"description\": \"Indicates if Quartz Crystal Microbalance was used.\"\n",
      "                }\n",
      "            }\n",
      "        }\n",
      "    },\n",
      "    \"required\": [\n",
      "        \"processParameters\",\n",
      "        \"materialProperties\",\n",
      "        \"reactionMechanisms\",\n",
      "        \"simulationMethods\",\n",
      "        \"characterizationTechniques\"\n",
      "    ]\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "print('Stage-2 Updated Schema:\\n')\n",
    "print(json.dumps(stage2_schema, indent=4))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "expert_feedback_stage2 = '1. Remove the reactor property in the additional details and include this in the reactor design property in the process conditions unit under the reactor name.\\n2. Bubblertemperatures: merge it with the precursor temperature property under the BubblerTemperature name in the precursor unit.\\n3. Substrate type and substrate units should be merged under the Substrate name'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 📄⛏️ Stage 03: Finalize Schema Refinement <a id='finalize-schema-refinement'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Similar to Stage 2, Stage 3 follows the same workflow and uses the same approach, with the key difference being the use of a larger corpus of scientific literature related to the ALD process. The goal of this stage is to provide the LLM with a broader perspective and more comprehensive information about the target process. The prompt used for this stage is provided [here](../src/prompts/prompt_template3.py)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "from prompts import prompt_template3\n",
    "\n",
    "def schema_extraction_stage3(llm_model_name: str, schema: dict, review: str, scientific_literature: list):\n",
    "    \"\"\"\n",
    "    Updates the JSON schema with the large scientific literature dataset and continous expert's feedback\n",
    "\n",
    "    Args:\n",
    "        llm_model_name (str): The large language model name\n",
    "        schema (dict): The JSON schema from stage 2\n",
    "        review (str): The expert's feedback on the schema from stage 2\n",
    "        scientific_literature (list): The list of text of large scientific literature dataset\n",
    "\n",
    "    Returns:\n",
    "        dict: The updated JSON schema\n",
    "    \"\"\"\n",
    "    \n",
    "    #Initializing the object for LLM Inference\n",
    "    llm = Openai_LLM_Inference(llm_model_name)\n",
    "    print(f'Using {llm}')\n",
    "    \n",
    "    #Initializing the counter\n",
    "    index = 0\n",
    "    \n",
    "    #Iterating over the scientific literature\n",
    "    for sci_lit in scientific_literature:\n",
    "        print(f'\\n========================== Iteration {index + 1} =================================')\n",
    "        \n",
    "        #Extracting the schema from the LLM\n",
    "        print('\\nCalling the completion API of the model')\n",
    "        var_dict = {'current_schema': schema, 'full_text': sci_lit, 'domain_expert_review': review}\n",
    "        model_output = llm.completion(prompt_template3, var_dict)\n",
    "\n",
    "        #Returns None, if any exception occurrs during the LLM Inference\n",
    "        if not model_output:\n",
    "            print(f'No output from model: {llm_model_name}, continuing to the next paper..')\n",
    "            continue\n",
    "        \n",
    "        #Extracting the updated schema\n",
    "        updated_schema = extract_json_schema(model_output, json_encl_expr = [('```json', '```'), ('```', '```')])\n",
    "        if not updated_schema:\n",
    "            print('Unable to extract JSON, stopping the inference from the LLM...')\n",
    "            break\n",
    "\n",
    "        #Updating the previous schema with the current schema for next iteration\n",
    "        schema = updated_schema\n",
    "        index += 1\n",
    "    \n",
    "    print(f'JSON schema updated with LLM: {llm_model_name}')\n",
    "    return schema"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using OPENAI - LLM Inference with Model: gpt-4o\n",
      "\n",
      "========================== Iteration 1 =================================\n",
      "\n",
      "Calling the completion API of the model\n",
      "\n",
      "========================== Iteration 2 =================================\n",
      "\n",
      "Calling the completion API of the model\n",
      "\n",
      "========================== Iteration 3 =================================\n",
      "\n",
      "Calling the completion API of the model\n",
      "JSON schema updated with LLM: gpt-4o\n"
     ]
    }
   ],
   "source": [
    "llm_model_name = 'gpt-4o'\n",
    "stage3_schema = schema_extraction_stage3(llm_model_name, stage2_schema, expert_feedback_stage2, stage_3_text[:5])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stage-3 Updated Schema:\n",
      "\n",
      "{\n",
      "    \"$schema\": \"http://json-schema.org/draft-07/schema#\",\n",
      "    \"title\": \"Atomic Layer Deposition (ALD) Process Schema\",\n",
      "    \"type\": \"object\",\n",
      "    \"properties\": {\n",
      "        \"processParameters\": {\n",
      "            \"type\": \"object\",\n",
      "            \"description\": \"Parameters defining the ALD process.\",\n",
      "            \"properties\": {\n",
      "                \"temperature\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Temperature at which the ALD process is conducted, in Celsius. Typical range is 50 to 500 \\u00b0C.\",\n",
      "                    \"units\": \"Celsius\"\n",
      "                },\n",
      "                \"pressure\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Pressure inside the reactor during the ALD process, typically maintained at 0.25 Torr.\",\n",
      "                    \"units\": \"Pascals\"\n",
      "                },\n",
      "                \"precursorFlowRate\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Flow rate of the precursor gas, in standard cubic centimeters per minute (sccm).\",\n",
      "                    \"units\": \"sccm\"\n",
      "                },\n",
      "                \"purgeFlowRate\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Flow rate of the purge gas, in standard cubic centimeters per minute (sccm).\",\n",
      "                    \"units\": \"sccm\"\n",
      "                },\n",
      "                \"substrateVelocity\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Velocity of the substrate during the ALD process, in meters per second.\",\n",
      "                    \"units\": \"m/s\"\n",
      "                },\n",
      "                \"gapDistance\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Distance between the substrate and the precursor source, in millimeters.\",\n",
      "                    \"units\": \"mm\"\n",
      "                },\n",
      "                \"precursorType\": {\n",
      "                    \"type\": \"string\",\n",
      "                    \"description\": \"Type of precursor used in the ALD process, e.g., metal amides, metal halides, alkyl amides.\"\n",
      "                },\n",
      "                \"precursorPulseTime\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Duration of the precursor pulse, in seconds.\",\n",
      "                    \"units\": \"seconds\"\n",
      "                },\n",
      "                \"coReactantPulseTime\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Duration of the co-reactant pulse, in seconds.\",\n",
      "                    \"units\": \"seconds\"\n",
      "                },\n",
      "                \"vaporPressure\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Vapor pressure of the precursor, in Pascals.\",\n",
      "                    \"units\": \"Pascals\"\n",
      "                },\n",
      "                \"reactivity\": {\n",
      "                    \"type\": \"string\",\n",
      "                    \"description\": \"Reactivity of the precursors with hydroxylated surfaces, influencing deposition efficiency.\"\n",
      "                },\n",
      "                \"plasmaSource\": {\n",
      "                    \"type\": \"boolean\",\n",
      "                    \"description\": \"Indicates if a plasma source is used in the ALD process.\"\n",
      "                },\n",
      "                \"reactorDesign\": {\n",
      "                    \"type\": \"object\",\n",
      "                    \"description\": \"Design of the reactor used in the ALD process.\",\n",
      "                    \"properties\": {\n",
      "                        \"reactorName\": {\n",
      "                            \"type\": \"string\",\n",
      "                            \"description\": \"Name of the reactor design.\"\n",
      "                        },\n",
      "                        \"plasmaConfiguration\": {\n",
      "                            \"type\": \"string\",\n",
      "                            \"description\": \"Configuration of the plasma used, e.g., radical-enhanced, direct, remote.\"\n",
      "                        }\n",
      "                    }\n",
      "                }\n",
      "            },\n",
      "            \"required\": [\n",
      "                \"temperature\",\n",
      "                \"pressure\",\n",
      "                \"precursorFlowRate\",\n",
      "                \"purgeFlowRate\"\n",
      "            ]\n",
      "        },\n",
      "        \"materialProperties\": {\n",
      "            \"type\": \"object\",\n",
      "            \"description\": \"Properties of the material being deposited.\",\n",
      "            \"properties\": {\n",
      "                \"growthRate\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Growth rate of the film, in nanometers per cycle.\",\n",
      "                    \"units\": \"nm/cycle\"\n",
      "                },\n",
      "                \"uniformity\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Uniformity of the film thickness across the substrate, as a percentage.\",\n",
      "                    \"units\": \"%\"\n",
      "                },\n",
      "                \"roughness\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Surface roughness of the deposited film, in nanometers.\",\n",
      "                    \"units\": \"nm\"\n",
      "                },\n",
      "                \"density\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Density of the deposited film, in grams per cubic centimeter.\",\n",
      "                    \"units\": \"g/cm^3\"\n",
      "                },\n",
      "                \"chemicalComposition\": {\n",
      "                    \"type\": \"string\",\n",
      "                    \"description\": \"Chemical composition of the deposited film.\"\n",
      "                },\n",
      "                \"electricalProperties\": {\n",
      "                    \"type\": \"object\",\n",
      "                    \"description\": \"Electrical properties of the deposited film.\",\n",
      "                    \"properties\": {\n",
      "                        \"leakageCurrentDensity\": {\n",
      "                            \"type\": \"number\",\n",
      "                            \"description\": \"Leakage current density of the film, in nanoamperes per square centimeter.\",\n",
      "                            \"units\": \"nA/cm^2\"\n",
      "                        },\n",
      "                        \"breakdownField\": {\n",
      "                            \"type\": \"number\",\n",
      "                            \"description\": \"Breakdown field of the film, in megavolts per centimeter.\",\n",
      "                            \"units\": \"MV/cm\"\n",
      "                        },\n",
      "                        \"dielectricConstant\": {\n",
      "                            \"type\": \"number\",\n",
      "                            \"description\": \"Dielectric constant of the film.\"\n",
      "                        }\n",
      "                    }\n",
      "                },\n",
      "                \"gasDiffusionProperties\": {\n",
      "                    \"type\": \"object\",\n",
      "                    \"description\": \"Gas diffusion properties of the film.\",\n",
      "                    \"properties\": {\n",
      "                        \"CO2LossRate\": {\n",
      "                            \"type\": \"number\",\n",
      "                            \"description\": \"CO2 loss rate through the film, in cubic centimeters per day per bar per square centimeter.\",\n",
      "                            \"units\": \"cm^3/(day bar cm^2)\"\n",
      "                        },\n",
      "                        \"barrierImprovementFactor\": {\n",
      "                            \"type\": \"number\",\n",
      "                            \"description\": \"Barrier improvement factor for gas diffusion.\"\n",
      "                        }\n",
      "                    }\n",
      "                },\n",
      "                \"conformality\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Degree of conformality of the film, as a percentage of step coverage.\",\n",
      "                    \"units\": \"%\"\n",
      "                },\n",
      "                \"filmOrientation\": {\n",
      "                    \"type\": \"string\",\n",
      "                    \"description\": \"Crystallographic orientation of the deposited film.\"\n",
      "                },\n",
      "                \"impurityContent\": {\n",
      "                    \"type\": \"object\",\n",
      "                    \"description\": \"Impurity content in the deposited film.\",\n",
      "                    \"properties\": {\n",
      "                        \"hydrogen\": {\n",
      "                            \"type\": \"number\",\n",
      "                            \"description\": \"Hydrogen content in atomic percent.\",\n",
      "                            \"units\": \"at.%\"\n",
      "                        },\n",
      "                        \"carbon\": {\n",
      "                            \"type\": \"number\",\n",
      "                            \"description\": \"Carbon content in atomic percent.\",\n",
      "                            \"units\": \"at.%\"\n",
      "                        },\n",
      "                        \"oxygen\": {\n",
      "                            \"type\": \"number\",\n",
      "                            \"description\": \"Oxygen content in atomic percent.\",\n",
      "                            \"units\": \"at.%\"\n",
      "                        }\n",
      "                    }\n",
      "                },\n",
      "                \"adhesion\": {\n",
      "                    \"type\": \"string\",\n",
      "                    \"description\": \"Adhesion quality of the film to the substrate.\"\n",
      "                },\n",
      "                \"refractiveIndex\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Refractive index of the film at a specified wavelength.\",\n",
      "                    \"units\": \"dimensionless\"\n",
      "                },\n",
      "                \"filmThickness\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Thickness of the deposited film, in nanometers.\",\n",
      "                    \"units\": \"nm\"\n",
      "                },\n",
      "                \"porosity\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Porosity of the film, as a percentage.\",\n",
      "                    \"units\": \"%\"\n",
      "                },\n",
      "                \"resistivity\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Resistivity of the film, in microohm-centimeters.\",\n",
      "                    \"units\": \"\\u00b5\\u03a9\\u00b7cm\"\n",
      "                }\n",
      "            },\n",
      "            \"required\": [\n",
      "                \"growthRate\",\n",
      "                \"uniformity\",\n",
      "                \"roughness\",\n",
      "                \"density\"\n",
      "            ]\n",
      "        },\n",
      "        \"reactionMechanisms\": {\n",
      "            \"type\": \"object\",\n",
      "            \"description\": \"Details of the reaction mechanisms involved in the ALD process.\",\n",
      "            \"properties\": {\n",
      "                \"chemisorptionMechanism\": {\n",
      "                    \"type\": \"string\",\n",
      "                    \"description\": \"Mechanism of chemisorption during the ALD process.\"\n",
      "                },\n",
      "                \"bindingAffinity\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Binding affinity of the adsorbate, in kilojoules per mole.\",\n",
      "                    \"units\": \"kJ/mol\"\n",
      "                },\n",
      "                \"stickingCoefficient\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Sticking coefficient of the adsorbate.\"\n",
      "                },\n",
      "                \"surfaceCoverage\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Surface coverage by the adsorbate, as a percentage.\",\n",
      "                    \"units\": \"%\"\n",
      "                }\n",
      "            },\n",
      "            \"required\": [\n",
      "                \"chemisorptionMechanism\",\n",
      "                \"bindingAffinity\",\n",
      "                \"stickingCoefficient\"\n",
      "            ]\n",
      "        },\n",
      "        \"simulationMethods\": {\n",
      "            \"type\": \"array\",\n",
      "            \"description\": \"Simulation methods used to model the ALD process.\",\n",
      "            \"items\": {\n",
      "                \"type\": \"string\",\n",
      "                \"enum\": [\n",
      "                    \"DFT\",\n",
      "                    \"kMC\",\n",
      "                    \"MD\",\n",
      "                    \"CFD\",\n",
      "                    \"LBM\",\n",
      "                    \"GCM\",\n",
      "                    \"CAMD\",\n",
      "                    \"HF\",\n",
      "                    \"ML\"\n",
      "                ]\n",
      "            }\n",
      "        },\n",
      "        \"characterizationTechniques\": {\n",
      "            \"type\": \"object\",\n",
      "            \"description\": \"Techniques used to characterize the ALD process and film properties.\",\n",
      "            \"properties\": {\n",
      "                \"FTIR\": {\n",
      "                    \"type\": \"boolean\",\n",
      "                    \"description\": \"Indicates if Fourier Transform Infrared Spectroscopy was used.\"\n",
      "                },\n",
      "                \"QCM\": {\n",
      "                    \"type\": \"boolean\",\n",
      "                    \"description\": \"Indicates if Quartz Crystal Microbalance was used.\"\n",
      "                }\n",
      "            }\n",
      "        },\n",
      "        \"precursor\": {\n",
      "            \"type\": \"object\",\n",
      "            \"description\": \"Details about the precursor used in the ALD process.\",\n",
      "            \"properties\": {\n",
      "                \"BubblerTemperature\": {\n",
      "                    \"type\": \"number\",\n",
      "                    \"description\": \"Temperature of the precursor bubbler, in Celsius.\",\n",
      "                    \"units\": \"Celsius\"\n",
      "                }\n",
      "            }\n",
      "        },\n",
      "        \"substrate\": {\n",
      "            \"type\": \"object\",\n",
      "            \"description\": \"Details about the substrate used in the ALD process.\",\n",
      "            \"properties\": {\n",
      "                \"Substrate\": {\n",
      "                    \"type\": \"string\",\n",
      "                    \"description\": \"Type and units of the substrate used in the ALD process.\"\n",
      "                }\n",
      "            }\n",
      "        }\n",
      "    },\n",
      "    \"required\": [\n",
      "        \"processParameters\",\n",
      "        \"materialProperties\",\n",
      "        \"reactionMechanisms\",\n",
      "        \"simulationMethods\",\n",
      "        \"characterizationTechniques\"\n",
      "    ]\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "print('Stage-3 Updated Schema:\\n')\n",
    "print(json.dumps(stage3_schema, indent=4))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "awases",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}