Skip to content

Tutorial: Zero to Prediction

This guide will take you from a fresh repository clone to running your first machine learning inference using GoServe and Docker.

1. Prerequisites

Ensure you have the following installed: - Docker - Git - Python 3.x (only for the initial model download)

2. Clone and Setup

First, clone the repository and navigate to the project root:

git clone https://github.com/goserve-run/goserve.git
cd goserve

3. Build the GoServe Image

Building the image handles all C++ dependencies (ONNX Runtime) automatically.

docker build -t goserve .

4. Prepare the Sample Model

We provide a Python script to download a pre-trained MobileNetV2 model.

cd examples/image-classification
python download_model.py
cd ../..

This creates the examples/image-classification/models/mobilenetv2.onnx file.

5. Launch the Server

Run GoServe in a Docker container, mounting the examples directory so the server can access the model.

# Windows (PowerShell)
docker run -d --name goserve -p 8080:8080 -v "$(Get-Location)/examples:/app/examples" goserve

# Linux / macOS
docker run -d --name goserve -p 8080:8080 -v "$(pwd)/examples:/app/examples" goserve

6. Load the Model

Send a request to GoServe to load the MobileNet model into memory. GoServe will automatically introspect the model to identify its input and output nodes.

curl -X POST http://localhost:8080/v1/models \
  -H "Content-Type: application/json" \
  -d '{
    "name": "mobilenet", 
    "path": "/app/examples/image-classification/models/mobilenetv2.onnx"
  }'

7. Run Your First Prediction

We will use a small Python script to preprocess an image and send it to the server.

cd examples/image-classification
# Create a virtual environment and install dependencies
python -m venv venv
.\venv\Scripts\activate  # Linux/macOS: source venv/bin/activate
pip install -r requirements.txt

# Download a sample image
python -c "import urllib.request; urllib.request.urlretrieve('https://raw.githubusercontent.com/pytorch/hub/master/images/dog.jpg', 'dog.jpg')"

# Run inference
python test_classification.py dog.jpg

Congratulations! You have successfully deployed a high-performance inference server and classified your first image.

Next Steps

  • Explore the API Reference to learn about generic tensor support.
  • Check the Benchmarks to see how GoServe compares to Python servers.
  • Configure monitoring using the Metrics Guide.