#llm

Articles tagged with #llm

Deploying Portkey - Open-Source AI Gateway for LLM Routing
Deploy Portkey on a Linux server with Docker Compose and Traefik, securing an OpenAI-compatible AI gateway with Basic Auth and an IP allowlist.
Sep 30, 20265 min read3
Deploying Flowise - Open-Source Visual LLM Workflow Builder
Deploy Flowise with Docker Compose and Traefik, then build and test a chatflow combining a prompt, chat model, and memory.
Sep 30, 20268 min read1
Deploying LiteLLM: An Open-Source AI Gateway
Deploy LiteLLM on a Linux server with Docker Compose, PostgreSQL, Prometheus, and Traefik to run an OpenAI-compatible AI gateway.
Sep 23, 202610 min read2
Deploying Inference Using NVIDIA Dynamo and vLLM
Deploy NVIDIA Dynamo with vLLM for high-throughput, low-latency LLM inference using aggregated and disaggregated GPU serving.
Sep 3, 20269 min read
Installing LM Studio – A Graphical Application for Running LLMs
Install LM Studio on Linux, run it as a system service, download and chat with local models, and expose its API through Nginx with TLS.
Aug 26, 20264 min read1
Deploying DeepSeek R1 Reasoning LLM Using SGLang
Deploy DeepSeek R1 for reasoning inference on an AMD Instinct MI300X GPU server using a ROCm-supported SGLang container, then test it with an HTTP request.
Aug 26, 20262 min read