Claude skills for creating a MWC module

Initial prompt

❯ We are going to create a new module, mwc2/unit3/lab_local_models. The curriculum page for the lab does not yet exist; the repo for the module exists but is a stub. The learning objectives of this lab are:

  1. Students understand how to host LLMs locally and to call them from their own code.
  2. Students understand how to provide Python functions as tools to LLMs.
    • As a subtopic, we will need to introduce type signatures. So far, we have discussed types somewhat (e.g. mwc1/unit2/problemset_numberwords/), but have not used any strongly-typed interfaces.
  3. Students understand how to browse availalbe models and select one with the desired charactersistics, which will run on their device.
  4. Students understand the basics of quantization and distillation, techniques by which LLMs parameters are compressed while preserving as much performance as possible.

The repo code should contain a script goodmorning.py, which imports ollama, loads a very lightweight model (which will still work on an older raspberry pi), and implements a simple CLI chat interface. All the code should be written for absolute beginners in mind, should not contain comments and only brief docstrings (more lines of text adds cognitive load for beginners), and should not use type hints. Decompose liberally into functions to reduce complexity.

The curriculum page should start by explaining the goals of this lab, then should walk students through importing ollama, loading a model (again, very lightweight), and interacting with the model via the chat interface. Introduce the idea of a system prompt. Then, the first checkpoint should have students create an app which helps them get ready for school in the morning. The app should have a system propmt which encodes information about the user’s morning routines.

Next, introduce tools. There should be another module in the lab repo, tools.py, which defines day_of_week, a function which returns the day of the week. Explain how this works in the lab, and guide students through updating chat.py to import tools.day_of_week and passing it to ollama as a tool. Students should update the system prompt to tell it to make use of the day_of_week tool. As part of explaining this, you need to create an aside introducing type hints. Explain that they are needed for the LLM to understand how to use the function; also note the importance of the docstring (use the Google docstring format, and tell students to follow the pattern). Add another function to tools, local_weather, which fetches a local weather report (follow the pattern used in a previous lab, /Users/chrisp/Repos/MWC/modules/lab_weather). For the second checkpoint, students should pass day_of_week and local_weather as tools and update the system prompt to use them to start the chat and contextualize the response.

Next, create a markdown file in the lab repo called analysis.md. In the next section of the lab, introduce quantization, distillation, and any other relevant techniques for reducing the size and memory footprint of model weights. Explain how students can browse models on huggingface, and show them how to check how much memory is availalbe on their system. Explain the role of a GPU in speeding up model inference. Then, in analysis.md, create a list of questions students should answer:

  1. How much memory is availalbe on your system?
  2. List two models from hugging face which could run on your system. Explain what each of them is good at.
  3. Choose an interesting-looking model which cannot run on your system. What is this model good at doing? Imagine an AI-powered app you could create with this model.
  4. Ollama can easily be configured to use a remotely-hosted model; you just provide the URL where the model is hosted. List some of the advantages of locally-hosted models, and some of the advantages of remotely-hosted models. What kinds of apps would be most suitable for locally-hosted models? What kinds of apps would be most suitable for remotely-hosted models?

The deliverables at the end of the lab are to answer these questions.

Revisions

❯ Minor updates:

  • Make the SYSTEM_PROMPT variable a triple-quoted string, so students can add lots more text without having to change the syntax. (change the repo code and the lab page.) Also add notes that the system prompt can be very long. As an aside, note that AI services sometimes try to keep their system prompts secret, because they contain rules governing the agent’s behavior.
  • Instead of import tools and then referring to scoped functions (e.g. tools.day_of_week), use from tools import day_of_week, local_weather. Make this change in both repo and lab page.
  • Simplify the code in run_tools; you are using constructs students have not seen. Make the TOOLS variable a dict. If possible, avoid using the ** operator when calling the tool function with the provided arguments. If we can avoid the note about name, remove it. Make this change in the repo and the lab page.
  • Update the code in section 3 (Tools) so that it only walks the student through how to use the day_of_week tool. The student will need to figure out how to add local_weather themselves.
  • In part 4 (Choosing a model), focus on two high-level questions: what is the model good at (discuss how the training data and task will affect what the model is good at, and give a few examples of models trained for different skills), and how much memory does the model require? The discussion of quantization and distillation should go under the latter question.
  • Rename analysis.md to questions.md. Show the questions in the deliverables section.
  • Add a .commit_template to the repo, which asks the student to list five ideas for apps they would like to build with LLMs.