🚧 Cortex.cpp is currently under development. Our documentation outlines the intended behavior of Cortex, which may not yet be fully implemented in the codebase.
Proxy Configuration Guide
This document describes how to configure proxy settings for Cortex to be able to work behind your proxy.
Command Line Interface (CLI)​
Basic Usage​
cortex config [OPTIONS] [COMMAND]
Commands​
status: Display all current configurations
cortex config status
Example Output:
+-----------------------+------------------------+| Config name | Value |+-----------------------+------------------------+| no_proxy | localhost,127.0.0.1 |+-----------------------+------------------------+| proxy_password | |+-----------------------+------------------------+| proxy_url | http://localhost:8080 |+-----------------------+------------------------+| proxy_username | |+-----------------------+------------------------+| verify_host_ssl | true |+-----------------------+------------------------+| verify_peer_ssl | false |+-----------------------+------------------------+| verify_proxy_host_ssl | true |+-----------------------+------------------------+| verify_proxy_ssl | true |+-----------------------+------------------------+
Options​
| Option | Description | Example |
|---|---|---|
-h, --help | Print help message and exit | |
--proxy_url <proxy_url> | Set the proxy URL | cortex config --proxy_url http://localhost:8080 |
--proxy_username <proxy_username> | Set the username for proxy | cortex config --proxy_username my_username |
--proxy_password <proxy_password> | Set the password for proxy | cortex config --proxy_password my_password |
--no_proxy <no_proxy> | Set the no_proxy list | cortex config --no_proxy localhost,127.0.0.1 |
--verify_proxy_ssl [on/off] | Verify proxy SSL | cortex config --verify_proxy_ssl on |
--verify_proxy_host_ssl [on/off] | Verify proxy host SSL | cortex config --verify_proxy_host_ssl on |
--verify_peer_ssl [on/off] | Verify peer SSL | cortex config --verify_peer_ssl off |
--verify_host_ssl [on/off] | Verify host SSL | cortex config --verify_host_ssl on |
Proxy API Configuration​
Endpoints​
Get Current Configuration​
GET /v1/configs
Retrieves the current configuration settings.
Response​
{ "allowed_origins": [ "http://localhost:39281", "http://127.0.0.1:39281", "http://0.0.0.0:39281" ], "cors": true, "no_proxy": "localhost,127.0.0.1", "proxy_password": "", "proxy_url": "http://localhost:8080", "proxy_username": "", "verify_host_ssl": true, "verify_peer_ssl": false, "verify_proxy_host_ssl": true, "verify_proxy_ssl": true}
Update Configuration​
PATCH /v1/configs
Updates proxy configuration settings.
Request Headers​
Content-Type: application/json
Request Body​
{ "no_proxy": "localhost", "proxy_url": "http://localhost:8080", "proxy_username": "my_username", "proxy_password": "my_password", "verify_host_ssl": false, "verify_peer_ssl": false, "verify_proxy_host_ssl": false, "verify_proxy_ssl": false}
Parameters​
| Field | Type | Description |
|---|---|---|
no_proxy | string | List of origins which request do not need to go through a proxy. Comma separated value |
proxy_url | string | Proxy URL |
proxy_username | string | Username for proxy authentication |
proxy_password | string | Password for proxy authentication |
verify_host_ssl | boolean | Verify host SSL |
verify_peer_ssl | boolean | Verify peer SSL |
verify_proxy_host_ssl | boolean | Verify proxy host SSL |
verify_proxy_ssl | boolean | Verify proxy SSL |
Response​
{ "config": { "allowed_origins": [ "http://localhost:39281", "http://127.0.0.1:39281", "http://0.0.0.0:39281" ], "cors": true, "no_proxy": "localhost", "proxy_password": "my_password", "proxy_url": "http://localhost:8080", "proxy_username": "my_username", "verify_host_ssl": false, "verify_peer_ssl": false, "verify_proxy_host_ssl": false, "verify_proxy_ssl": false }, "message": "Configuration updated successfully"}
Testing proxy configuration​
You can test your proxy configuration using mitmproxy. This guide is written on macOS, but you can use it on any other platform.
Install mitmproxy​
brew install mitmproxy
Start mitmproxy​
mitmproxy --set stream_large_bodies=1m
mitmproxy will start on port 8080. After mitmproxy started, you can adding options by pressing O. mitmproxy will display an option screen. You can check their document to learn more about mitmproxy. But let's take a simple option for now by setting the proxyauth for our local proxy. Inside the option screen, search for proxyauth and hit enter. Then, type username:password and hit enter again. You will see your newly added option is red-colored.
Configuring Cortex to use that proxy​
Let's using CLI to configure Cortex to use that proxy.
cortex config --proxy_url http://localhost:8080 --proxy_username username --proxy_password password
Testing the proxy​
Now, let's test the proxy. If you are setting the username and password correctly (same with proxyauth in mitmproxy), you will see the request in mitmproxy. For example, command cortex pull tinyllama should be successfully and returns a list of selectable models. Also, you will see your request in mitmproxy CLI screen.
Let's try to use a wrong authentication for your proxy.
cortex config --proxy_password wrong_pw
Now, let's test the proxy again. You will see the request is failed and returns an error.