DSS’s Model cache¶

The model cache¶

DSS has its own (optional) managed cache to store models from HuggingFace.

If enabled at the connection level, the cache is automatically populated when using the LLM Mesh with pre-trained models from HuggingFace. Models are downloaded from HuggingFace hub.

View the model cache content in: Settings > Misc. You can also delete, export or import models.

Import and export models¶

If your DSS instance does not have access to HuggingFace (huggingface.co), you can manually import a model archive, typically one exported from the model cache of another DSS design or automation node with network access.

Build your own model archive to import¶

Note

It is simpler (and recommended) to import a model that was previously exported by DSS when that is possible, instead of creating an archive manually.

If you want to manually create an archive it should contain the following structure:

a root folder
- a folder named “model” containing the HuggingFace model repo content
- a file named “model_metadata.json”

To retrieve the model folder content from HuggingFace hub:

git lfs install
git clone https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2

Example of a model archive content:

sentence-transformers_2fall-MiniLM-L6-v2 ← folder at the root (its name is not important)
├── model ← a folder named "model" containing the HuggingFace model repo content
│   ├── 1_Pooling
│   │   └── config.json
│   ├── README.md
│   ├── config.json
│   ├── config_sentence_transformers.json
│   ├── data_config.json
│   ├── modules.json
│   ├── pytorch_model.bin
│   ├── sentence_bert_config.json
│   ├── special_tokens_map.json
│   ├── tokenizer.json
│   ├── tokenizer_config.json
│   ├── train_script.py
│   └── vocab.txt
└── model_metadata.json ← a file named "model_metadata.json"

The model_metadata.json file should have the following schema:

{
    "commitHash": "7dbbc90392e2f80f3d3c277d6e90027e55de9125",
    "downloadDate": 1698300636139,
    "downloadedBy": "admin",
    "lastDssUsage": 1699570884724,
    "lastModified": "2022-11-07T08:44:33.000Z",
    "lastUsedBy": "admin",
    "libraryName": "sentence-transformers",
    "modelDefinition": {
        "folderName": "sentence-transformers_2fall-MiniLM-L6-v2",
        "key": "hf@sentence-transformers/all-MiniLM-L6-v2",
        "modelId": "sentence-transformers/all-MiniLM-L6-v2",
        "revision": "main"
    },
    "pipelineName": "sentence-similarity",
    "sizeInBytes": 91652688,
    "taggedLanguages": [
        "en"
    ],
    "tags": [
        "sentence-transformers",
        "pytorch",
        "tf",
        "rust",
        "bert",
        "feature-extraction",
        "sentence-similarity",
        "en",
        "dataset:s2orc",
        "dataset:flax-sentence-embeddings/stackexchange_xml",
        ...
        "arxiv:2104.08727",
        "arxiv:1704.05179",
        "arxiv:1810.09305",
        "license:apache-2.0",
        "endpoints_compatible",
        "has_space",
        "region:us"
    ],
    "url": "https://huggingface.co/sentence-transformers%2Fall-MiniLM-L6-v2/tree/main",
    "version": 0
}

Most of these fields can be retrieved from the HuggingFace model repository.

The important ones are:

modelDefinition
- revision: the targeted git revision used to retrieve this model (main by default), it can be a branch or a commit hash
- modelId: the model name
- key: consists of hf@<modelId> or hf@<modelId>@<revision> if a specific revision was used
- folderName: consists of modelId or modelId@revision the modelId and revision need to be escaped. Only alphanumerical values and dashes are kept. _xx represent each UTF-8 escaped byte.
version: as of now should be 0
url: the url used to fetch the model