deploy: 7b19b76

ErikBjare · Oct 20, 2024 · 98c32cf · 98c32cf
1 parent 98b84db
commit 98c32cf
Show file tree

Hide file tree

Showing 5 changed files with 36 additions and 34 deletions.
diff --git a/docs/arewetiny.html b/docs/arewetiny.html
@@ -410,15 +410,15 @@ <h1>Are we tiny?<a class="headerlink" href="#are-we-tiny" title="Link to this he
 <h2>Startup time<a class="headerlink" href="#startup-time" title="Link to this heading">#</a></h2>
 <div class="highlight-text notranslate"><div class="highlight"><pre><span></span>$ make bench-importtime
 ...
-     267777 |       gptme.commands
-     272076 |     gptme.chat
-     289194 |   gptme.cli
-     336729 | gptme
-     356860 | openai
-
-real	0m1.430s
-user	0m1.248s
-sys	0m0.189s
+     265142 |       gptme.commands
+     269578 |     gptme.chat
+     286525 |   gptme.cli
+     333482 | gptme
+     362806 | openai
+
+real	0m1.442s
+user	0m1.265s
+sys	0m0.190s
 make[2]: Leaving directory &#39;/home/runner/work/gptme/gptme&#39;
 </pre></div>
 </div>
@@ -430,19 +430,20 @@ <h3>LoC Core<a class="headerlink" href="#loc-core" title="Link to this heading">
 <div class="highlight-text notranslate"><div class="highlight"><pre><span></span>$ make cloc-core
 make[2]: Entering directory &#39;/home/runner/work/gptme/gptme&#39;
 cloc gptme/*.py gptme/tools/__init__.py gptme/tools/base.py --by-file
-      26 text files.
-classified 26 files
-      26 unique files.                              
+      27 text files.
+classified 27 files
+      27 unique files.                              
        0 files ignored.
 
-github.com/AlDanial/cloc v 1.90  T=0.03 s (806.9 files/s, 127218.6 lines/s)
+github.com/AlDanial/cloc v 1.90  T=0.03 s (804.6 files/s, 130368.0 lines/s)
 ----------------------------------------------------------------------------------------
 File                                      blank        comment           code
 ----------------------------------------------------------------------------------------
 gptme/chat.py                                50             73            271
 gptme/logmanager.py                          63             45            270
 gptme/cli.py                                 48             38            251
 gptme/message.py                             51             62            251
+gptme/ncurses.py                             37              0            239
 gptme/util.py                                47             24            236
 gptme/commands.py                            18             16            188
 gptme/llm.py                                 38             28            153
@@ -466,7 +467,7 @@ <h3>LoC Core<a class="headerlink" href="#loc-core" title="Link to this heading">
 gptme/__version__.py                          1              0              5
 gptme/__main__.py                             1              0              3
 ----------------------------------------------------------------------------------------
-SUM:                                        653            603           2843
+SUM:                                        690            603           3082
 ----------------------------------------------------------------------------------------
 make[2]: Leaving directory &#39;/home/runner/work/gptme/gptme&#39;
 </pre></div>
@@ -482,7 +483,7 @@ <h3>LoC Tools<a class="headerlink" href="#loc-tools" title="Link to this heading
       16 unique files.                              
        0 files ignored.
 
-github.com/AlDanial/cloc v 1.90  T=0.02 s (774.1 files/s, 120857.4 lines/s)
+github.com/AlDanial/cloc v 1.90  T=0.02 s (766.7 files/s, 119699.0 lines/s)
 -------------------------------------------------------------------------------------
 File                                              blank        comment           code
 -------------------------------------------------------------------------------------
@@ -519,7 +520,7 @@ <h3>LoC Server<a class="headerlink" href="#loc-server" title="Link to this headi
        7 unique files.                              
        0 files ignored.
 
-github.com/AlDanial/cloc v 1.90  T=0.01 s (769.2 files/s, 93187.2 lines/s)
+github.com/AlDanial/cloc v 1.90  T=0.01 s (748.6 files/s, 90685.3 lines/s)
 ---------------------------------------------------------------------------------
 File                                          blank        comment           code
 ---------------------------------------------------------------------------------
@@ -547,7 +548,7 @@ <h3>LoC Tests<a class="headerlink" href="#loc-tests" title="Link to this heading
       18 unique files.                              
        0 files ignored.
 
-github.com/AlDanial/cloc v 1.90  T=0.02 s (1149.8 files/s, 84573.4 lines/s)
+github.com/AlDanial/cloc v 1.90  T=0.02 s (1191.2 files/s, 87618.5 lines/s)
 ------------------------------------------------------------------------------------------
 File                                        blank        comment           code
 ------------------------------------------------------------------------------------------
@@ -586,7 +587,7 @@ <h3>LoC Eval<a class="headerlink" href="#loc-eval" title="Link to this heading">
        8 unique files.                              
        0 files ignored.
 
-github.com/AlDanial/cloc v 1.90  T=0.01 s (660.1 files/s, 79793.3 lines/s)
+github.com/AlDanial/cloc v 1.90  T=0.01 s (656.4 files/s, 79340.6 lines/s)
 -------------------------------------------------------------------------------------
 File                                   blank        comment           code
 -------------------------------------------------------------------------------------
@@ -609,13 +610,13 @@ <h3>LoC Eval<a class="headerlink" href="#loc-eval" title="Link to this heading">
 <h3>LoC Total<a class="headerlink" href="#loc-total" title="Link to this heading">#</a></h3>
 <div class="highlight-text notranslate"><div class="highlight"><pre><span></span>$ make cloc-total
 make[2]: Entering directory &#39;/home/runner/work/gptme/gptme&#39;
-cloc gptme/eval/main.py gptme/eval/suites/init_projects.py gptme/eval/suites/basic.py gptme/eval/suites/__init__.py gptme/eval/suites/browser.py gptme/eval/run.py gptme/eval/execenv.py gptme/eval/__init__.py gptme/eval/types.py gptme/eval/__main__.py gptme/eval/agents.py gptme/eval/filestore.py gptme/llm.py gptme/__version__.py gptme/llm_openai_models.py gptme/commands.py gptme/init.py gptme/chat.py gptme/server/__init__.py gptme/server/api.py gptme/server/cli.py gptme/server/__main__.py gptme/constants.py gptme/message.py gptme/util.py gptme/__init__.py gptme/tools/vision.py gptme/tools/patch.py gptme/tools/chats.py gptme/tools/save.py gptme/tools/python.py gptme/tools/_browser_playwright.py gptme/tools/shell.py gptme/tools/gh.py gptme/tools/youtube.py gptme/tools/__init__.py gptme/tools/subagent.py gptme/tools/tmux.py gptme/tools/screenshot.py gptme/tools/base.py gptme/tools/browser.py gptme/tools/read.py gptme/reduce.py gptme/dirs.py gptme/models.py gptme/llm_openai.py gptme/codeblock.py gptme/tabcomplete.py gptme/cli.py gptme/logmanager.py gptme/__main__.py gptme/interrupt.py gptme/prompts.py gptme/config.py gptme/useredit.py gptme/llm_anthropic.py tests/test_prompts.py tests/test_codeblock.py tests/test_browser.py tests/test_tools_chats.py tests/test_cli.py tests/test_reduce.py tests/test_util.py tests/test_message.py tests/test_server.py tests/test_tools_shell.py tests/test_config.py tests/test_shell.py tests/test_logmanager.py tests/test_tabcomplete.py tests/test_tools_python.py tests/test_eval.py tests/test_tools_subagent.py tests/test_tools_patch.py scripts/list_user_messages.py scripts/describe_api.py scripts/summarize_project.py scripts/gpt_todoer.py scripts/train/collect.py scripts/shorten_details.py scripts/treeofthoughts.py --by-file
-      81 text files.
-classified 81 files
-      81 unique files.                              
+cloc gptme/eval/main.py gptme/eval/suites/init_projects.py gptme/eval/suites/basic.py gptme/eval/suites/__init__.py gptme/eval/suites/browser.py gptme/eval/run.py gptme/eval/execenv.py gptme/eval/__init__.py gptme/eval/types.py gptme/eval/__main__.py gptme/eval/agents.py gptme/eval/filestore.py gptme/llm.py gptme/__version__.py gptme/llm_openai_models.py gptme/commands.py gptme/init.py gptme/chat.py gptme/server/__init__.py gptme/server/api.py gptme/server/cli.py gptme/server/__main__.py gptme/ncurses.py gptme/constants.py gptme/message.py gptme/util.py gptme/__init__.py gptme/tools/vision.py gptme/tools/patch.py gptme/tools/chats.py gptme/tools/save.py gptme/tools/python.py gptme/tools/_browser_playwright.py gptme/tools/shell.py gptme/tools/gh.py gptme/tools/youtube.py gptme/tools/__init__.py gptme/tools/subagent.py gptme/tools/tmux.py gptme/tools/screenshot.py gptme/tools/base.py gptme/tools/browser.py gptme/tools/read.py gptme/reduce.py gptme/dirs.py gptme/models.py gptme/llm_openai.py gptme/codeblock.py gptme/tabcomplete.py gptme/cli.py gptme/logmanager.py gptme/__main__.py gptme/interrupt.py gptme/prompts.py gptme/config.py gptme/useredit.py gptme/llm_anthropic.py tests/test_prompts.py tests/test_codeblock.py tests/test_browser.py tests/test_tools_chats.py tests/test_cli.py tests/test_reduce.py tests/test_util.py tests/test_message.py tests/test_server.py tests/test_tools_shell.py tests/test_config.py tests/test_shell.py tests/test_logmanager.py tests/test_tabcomplete.py tests/test_tools_python.py tests/test_eval.py tests/test_tools_subagent.py tests/test_tools_patch.py scripts/list_user_messages.py scripts/describe_api.py scripts/summarize_project.py scripts/gpt_todoer.py scripts/train/collect.py scripts/shorten_details.py scripts/treeofthoughts.py --by-file
+      82 text files.
+classified 82 files
+      82 unique files.                              
        0 files ignored.
 
-github.com/AlDanial/cloc v 1.90  T=0.07 s (1196.0 files/s, 144196.9 lines/s)
+github.com/AlDanial/cloc v 1.90  T=0.07 s (1164.6 files/s, 142617.9 lines/s)
 -------------------------------------------------------------------------------------
 File                                              blank        comment           code
 -------------------------------------------------------------------------------------
@@ -626,6 +627,7 @@ <h3>LoC Total<a class="headerlink" href="#loc-total" title="Link to this heading
 gptme/tools/shell.py                                 58             63            264
 gptme/cli.py                                         48             38            251
 gptme/message.py                                     51             62            251
+gptme/ncurses.py                                     37              0            239
 gptme/util.py                                        47             24            236
 tests/test_cli.py                                    72             41            236
 gptme/commands.py                                    18             16            188
@@ -701,7 +703,7 @@ <h3>LoC Total<a class="headerlink" href="#loc-total" title="Link to this heading
 gptme/server/__init__.py                              2              3              3
 gptme/server/__main__.py                              1              0              3
 -------------------------------------------------------------------------------------
-SUM:                                               1761           1603           6402
+SUM:                                               1798           1603           6641
 -------------------------------------------------------------------------------------
 make[2]: Leaving directory &#39;/home/runner/work/gptme/gptme&#39;
 </pre></div>

diff --git a/docs/evals.html b/docs/evals.html
@@ -423,18 +423,18 @@ <h2>Results<a class="headerlink" href="#results" title="Link to this heading">#<
 <div class="highlight-text notranslate"><div class="highlight"><pre><span></span>$ gptme-eval eval_results/*/eval_results.csv
 Model                                     hello           hello-patch     hello-ask       prime100        init-git         init-rust      whois-superuserlabs-ceo
 ----------------------------------------  --------------  --------------  --------------  --------------  ---------------  -------------  -------------------------
-anthropic/claude-3-5-sonnet-20240620      ✅ 23/23 631tk  ✅ 23/23 538tk  ✅ 23/23 707tk  ✅ 23/23 865tk  ✅ 23/23 1080tk  ✅ 4/4 1504tk  🔶 3/4 1306tk
-anthropic/claude-3-haiku-20240307         ✅ 23/23 384tk  ✅ 23/23 391tk  ✅ 23/23 434tk  ❌ 4/23 710tk   🔶 16/23 831tk   🔶 3/4 670tk   ✅ 4/4 1535tk
+anthropic/claude-3-5-sonnet-20240620      ✅ 24/24 629tk  ✅ 24/24 536tk  ✅ 24/24 708tk  ✅ 24/24 869tk  ✅ 24/24 1086tk  ✅ 4/4 1504tk  🔶 3/4 1306tk
+anthropic/claude-3-haiku-20240307         ✅ 24/24 384tk  ✅ 24/24 393tk  ✅ 24/24 434tk  🔶 5/24 752tk   🔶 17/24 824tk   🔶 3/4 670tk   ✅ 4/4 1535tk
 openai/gpt-4-turbo                        ✅ 3/3 255tk    ✅ 3/3 312tk    ✅ 3/3 376tk    ✅ 3/3 527tk    ✅ 4/4 590tk     ✅ 4/4 784tk   ✅ 6/7 819tk
-openai/gpt-4o                             ✅ 24/24 277tk  ✅ 24/24 301tk  ✅ 24/24 367tk  🔶 14/24 438tk  ✅ 24/24 684tk   ✅ 5/5 663tk   ✅ 5/5 1253tk
-openai/gpt-4o-mini                        ✅ 25/25 260tk  ✅ 25/25 320tk  ✅ 25/25 379tk  ✅ 25/25 586tk  ✅ 21/25 749tk   ✅ 6/6 813tk   ✅ 6/6 951tk
+openai/gpt-4o                             ✅ 25/25 279tk  ✅ 25/25 300tk  ✅ 25/25 366tk  🔶 15/25 438tk  ✅ 25/25 692tk   ✅ 5/5 663tk   ✅ 5/5 1253tk
+openai/gpt-4o-mini                        ✅ 26/26 260tk  ✅ 26/26 319tk  ✅ 26/26 377tk  ✅ 26/26 586tk  ✅ 22/26 749tk   ✅ 6/6 813tk   ✅ 6/6 951tk
 openai/o1-mini                            ✅ 3/3 354tk    ✅ 3/3 431tk    ✅ 3/3 460tk    🔶 2/3 567tk    🔶 3/5 2222tk    🔶 1/5 1412tk  🔶 2/3 813tk
 openai/o1-preview                         ✅ 2/2 308tk    ✅ 2/2 570tk    🔶 1/2 549tk    ✅ 2/2 490tk    ✅ 3/3 823tk     ✅ 1/1 656tk   ✅ 1/1 1998tk
 google/gemini-flash-1.5                   ✅ 2/2 225tk    ✅ 2/2 401tk    ✅ 2/2 430tk    ❌ 0/2 296tk    ✅ 1/1 686tk     ❌ 0/1 661tk   ✅ 1/1 1014tk
 google/gemini-pro-1.5                     ✅ 1/1 341tk    ✅ 1/1 419tk    ✅ 1/1 456tk    ✅ 1/1 676tk    🔶 2/3 431tk     🔶 1/2 1016tk  ✅ 2/2 1308tk
 google/gemma-2-27b-it                     ✅ 1/1 288tk    ✅ 1/1 384tk    ✅ 1/1 446tk    ✅ 1/1 714tk    ✅ 1/1 570tk     ❌ 0/1 535tk   ❌ 0/1 235tk
 google/gemma-2-9b-it                      ❌ 0/2 186tk    ✅ 2/2 370tk    ✅ 2/2 368tk    ❌ 0/2 545tk    ✅ 1/1 492tk     ❌ 0/1 1730tk  ❌ 0/1 352tk
-meta-llama/llama-3.1-405b-instruct        🔶 15/24 187tk  🔶 17/24 427tk  🔶 16/24 326tk  🔶 19/24 391tk  🔶 10/24 342tk   🔶 2/5 255tk   ❌ 0/5 85tk
+meta-llama/llama-3.1-405b-instruct        🔶 15/25 179tk  🔶 17/25 410tk  🔶 17/25 334tk  ✅ 20/25 395tk  🔶 11/25 356tk   🔶 2/5 255tk   ❌ 0/5 85tk
 meta-llama/llama-3.1-70b-instruct         ✅ 5/6 367tk    ✅ 5/6 424tk    ✅ 6/6 452tk    🔶 2/6 546tk    ✅ 5/6 813tk     🔶 3/4 682tk   🔶 2/3 1461tk
 meta-llama/llama-3.1-8b-instruct          ✅ 1/1 277tk    ✅ 1/1 441tk    ❌ 0/1 400tk    ❌ 0/1 5095tk   ✅ 1/1 2266tk    ❓ N/A         ❓ N/A
 meta-llama/llama-3.2-11b-vision-instruct  ✅ 2/2 352tk    ✅ 2/2 493tk    ❌ 0/2 479tk    ✅ 2/2 2643tk   ❓ N/A           ❓ N/A         ❓ N/A

diff --git a/docs/prompts.html b/docs/prompts.html
@@ -602,10 +602,10 @@ <h1>Prompts<a class="headerlink" href="#prompts" title="Link to this heading">#<
 
 
 These programs are available, among others:
-<span class="k">-</span><span class="w"> </span>git
 <span class="k">-</span><span class="w"> </span>apt-get
-<span class="k">-</span><span class="w"> </span>docker
 <span class="k">-</span><span class="w"> </span>pandoc
+<span class="k">-</span><span class="w"> </span>git
+<span class="k">-</span><span class="w"> </span>docker
 
 <span class="gu">### Examples</span>
 

diff --git a/docs/searchindex.js b/docs/searchindex.js
diff --git a/docs/tools.html b/docs/tools.html
@@ -449,10 +449,10 @@ <h2>Shell<a class="headerlink" href="#shell" title="Link to this heading">#</a><
 
 
 These programs are available, among others:
-<span class="k">-</span><span class="w"> </span>git
 <span class="k">-</span><span class="w"> </span>apt-get
-<span class="k">-</span><span class="w"> </span>docker
 <span class="k">-</span><span class="w"> </span>pandoc
+<span class="k">-</span><span class="w"> </span>git
+<span class="k">-</span><span class="w"> </span>docker
 </pre></div>
 </div>
 <p class="rubric">Examples</p>