metadata
license: gemma
base_model: google/functiongemma-270m-it
tags:
- function-calling
- litert
- on-device
- gemma
- infrastructure
- LiteRT-LM
datasets:
- custom
language:
- en
pipeline_tag: text-generation
FunctionGemma Infrastructure LiteRT-LM
A fine-tuned FunctionGemma-270M model converted to LiteRT-LM format for on-device inference. Designed for self-healing infrastructure and automatic error remediation.
Model Details
| Property | Value |
|---|---|
| Base Model | google/functiongemma-270m-it |
| Format | LiteRT-LM (.litertlm) |
| Quantization | Dynamic INT8 |
| File Size | 272 MB |
| Parameters | 270M |
Intended Use
This model is designed for dad-express, a self-healing gateway that monitors HTTP traffic and automatically fixes infrastructure configuration issues by calling the appropriate tools.
Supported Tools
The model was fine-tuned on 9 infrastructure tools:
| Tool | Description | Parameters |
|---|---|---|
addProxyRoute |
Add reverse proxy route | path, upstream, port |
addCorsHeaders |
Configure CORS headers | origin, credentials |
configureSsl |
Configure SSL certificate | hostname, selfSigned |
setEnvVariable |
Set environment variable | name, value |
exposePort |
Expose port in Docker/firewall | service, port |
addHostEntry |
Add hostname to /etc/hosts | hostname, ip |
restartService |
Restart a service | service |
clearCache |
Clear cache | cacheType |
modifyConfig |
Modify config file | file, key, value |
Training Details
Dataset
- Total Examples: 10,216
- Train/Eval Split: 90/10 (9,194 train, 1,022 eval)
- Format: Prompt-completion pairs using FunctionGemma chat template
- Distribution: ~1,000-1,200 examples per tool (balanced)
Training Configuration
Trained following Google's official FunctionGemma fine-tuning notebook:
SFTConfig(
num_train_epochs=2,
per_device_train_batch_size=4,
gradient_accumulation_steps=8,
learning_rate=1e-5,
lr_scheduler_type="cosine",
gradient_checkpointing=True,
packing=False,
optim="adamw_torch_fused",
bf16=True,
completion_only_loss=True, # Critical: only train on completion tokens
)
Hardware
- GPU: NVIDIA L4 (24GB)
- Training Time: ~23 minutes
- Conversion Time: ~10 minutes
Performance Metrics
| Metric | Value |
|---|---|
| Final Eval Loss | 0.034 |
| Token Accuracy | 98.6% |
| Training Steps | 576 |
LiteRT-LM Conversion
Converted using ai-edge-torch:
converter.convert_to_litert(
pytorch_model,
prefill_seq_len=256,
kv_cache_max_len=1024,
quantize="dynamic_int8",
output_format="litertlm",
)
LLM Metadata
start_token: { token_ids: { ids: [ 2 ] } }
stop_tokens: { token_str: "<end_of_turn>" }
stop_tokens: { token_str: "<start_function_response>" }
llm_model_type: { function_gemma: {} }
Usage
With LiteRT-LM Runtime
import { LiteRTLM } from 'litert-lm';
const model = await LiteRTLM.load('functiongemma-infrastructure_q8_ekv1024.litertlm');
const response = await model.generate(prompt);
Example Input/Output
Input:
Error: CORS - No 'Access-Control-Allow-Origin' header from http://localhost:3000
Output:
<start_function_call>call:addCorsHeaders{origin:<escape>http://localhost:3000<escape>,credentials:<escape>true<escape>}<end_function_call><start_function_response>
Dependencies
- Python 3.11+
- transformers==4.57.1
- trl==0.25.1
- datasets==4.4.1
- ai-edge-torch-nightly
- ai-edge-litert-nightly
License
This model inherits the Gemma Terms of Use.
Citation
@misc{functiongemma-infrastructure-litertlm,
title={FunctionGemma Infrastructure LiteRT-LM},
author={dad-express contributors},
year={2024},
publisher={Hugging Face},
url={https://huggingface.co/macmacmacmac/functiongemma-infrastructure-litertlm}
}