Learning Timeline
Key Insights
Hardware Requirements for Large Models
To run 70B models like DeepSeek smoothly, it is recommended to have at least 64GB of RAM (such as on a Mac Studio) to prevent memory swapping that could slow down performance.
Streaming API Support
Exo Labs now supports streaming output in Fabric. If text does not appear immediately, ensure your API endpoint is correctly configured to support 'stream: true'.
Benefits of Local Clusters
Using Exo Labs allows you to combine the processing power of multiple computers (Mac and Nvidia) to run models that are too large for a single machine.
Prompts
Creative Writing Test
Target:
Fabric CLI (via Exo Labs)
Tell me a story.
Text Summarization Test
Target:
Fabric CLI (via Exo Labs)
Summarize the following text and provide the main key points.
Step by Step
How to Integrate Exo Labs with Fabric CLI
- Ensure your Exo Labs cluster is active and the model (e.g., DeepSeek 70B) is fully loaded into RAM/GPU.
- Open Terminal and run the Fabric update process to ensure you have the latest version that supports Exo Labs.
- Run the command `fabric --setup` to start the API configuration.
- Select 'XO' or 'OpenAI-compatible API' from the list of providers.
- Enter the API endpoint URL provided by your Exo Labs cluster (usually the localhost address or the main host's local IP).
- Verify the configuration and save the settings.
- Test the connection by sending a simple text input using a pipe command to Fabric.
- Use the `-p` or `--pattern` flag followed by the Fabric pattern name (e.g., summarize) to start processing data.
