joungmin/life-helper
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[Developer] #218 Real Gemma 4 E2B integration via flutter_gemma 0.16.5

Browse Source 
Implements the OQ-1 follow-up to #215 v0.2.0: replace the placeholder
GemmaLlmService stub with a real flutter_gemma 0.16.5 backend driving
Gemma 4 E2B (litert-community/gemma-4-E2B-it-litert-lm, 2.41GB).

Highlights:
- GemmaLlmService.load → FlutterGemma.initialize + installModel.fromFile +
  getActiveModel; idempotent + FileSystemException on missing file.
- generateStructured uses Gemma 4 native function calling via
  createChat(tools: [Tool(...)], toolChoice: required). Stream parsed by
  collectFunctionCall — first FCR wins, ParallelFCR first-call wins,
  TextResponse/ThinkingResponse skipped, errors sanitized to prevent
  prompt leakage.
- main.dart wires _LazyLlmService adapter that resolves to GemmaLlmService
  when ModelLifecycle reports ready, MockLlmService otherwise.
- ai_providers.dart pins real model URL + SHA-256 (181938...39a63c).
- F2 hardening: ModelLifecycle.purge wraps each delete + meta remove in
  try/catch so a single OS-level flake cannot block opt-out.
- Android: INTERNET / FOREGROUND_SERVICE / POST_NOTIFICATIONS permissions
  + R8 proguard-rules.pro keeping MediaPipe / LiteRT / TFLite / protobuf
  JNI entry points (release builds otherwise crash on first inference).

Design-First: fn-gemma_llm_service.md updated to v2 — §C
(_appendSchemaInstruction) deprecated after reading flutter_gemma
0.16.5 source (Gemma 4 SDK injects tool declarations via template;
prompt-side append would double-wrap).

Tests:
- 10 new unit tests for collectFunctionCall covering all 8 fn-spec
  cases + 2 ParallelFunctionCallResponse paths.
- All 81 existing tests still pass.
- flutter analyze: 0 issues.

Refs #218

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
joungmin
2026-06-12 15:18:08 +09:00
parent a1f3c5f85d
commit 9a9eb2abd5
14 changed files with 646 additions and 175 deletions
Download Patch File Download Diff File Expand all files Collapse all files

150
app/lib/data/ai/gemma_llm_service.dart
View File

@@ -1,21 +1,40 @@
import 'dart:io';
import 'package:flutter/foundation.dart';
import 'package:flutter_gemma/flutter_gemma.dart';
import 'llm_service.dart';
/// Stub for the real `flutter_gemma` integration.
/// HuggingFace access token injected at build time via
/// `--dart-define=HF_TOKEN=hf_xxx`. Empty string is permitted —
/// flutter_gemma will only need it for the initial network download,
/// which our `ModelLifecycle` handles separately; activation from a
/// local file path generally does not require the token.
const String _hfToken = String.fromEnvironment('HF_TOKEN', defaultValue: '');
/// One-shot guard so [FlutterGemma.initialize] runs at most once per
/// isolate. Re-init is unsupported by the underlying plugin.
bool _initialized = false;
/// Real on-device LLM backend using flutter_gemma 0.16.5 + Gemma 4 E2B.
///
/// Wired up only after OQ-1 (exact model URL + SHA + flutter_gemma API
/// surface) is confirmed in Developer phase. Today this throws
/// `UnimplementedError` from every method — the rest of the system
/// (suggestFrame, ModelLifecycle, Riverpod providers) is built against the
/// `LlmService` abstract above and runs end-to-end with `MockLlmService`.
/// Wired into the existing #215 pipeline: `ModelLifecycle` downloads &
/// SHA-verifies the .litertlm file, then [load] registers that file with
/// flutter_gemma as the active model. [generateStructured] opens a
/// short-lived chat with a single [Tool] (Gemma 4 native function
/// calling) and returns the first matching [FunctionCallResponse]'s args.
///
/// When the package is added, replace the bodies with calls into
/// FlutterGemma.init / generateWithFunctionCalling per the package docs.
/// Existing tests + UI hooks remain unchanged.
/// Function-calling design notes (see fn-gemma_llm_service.md §B v2):
/// - Gemma 4 SDK injects the tool declaration via its chat template, so
/// we pass [Tool] to `createChat(tools: ...)` rather than appending a
/// schema instruction to the prompt (double-wrap risk).
/// - `ToolChoice.required` forces the model to emit a function call.
class GemmaLlmService implements LlmService {
final String modelPath;
GemmaLlmService({required this.modelPath});
InferenceModel? _model;
bool _loaded = false;
@override
@@ -23,14 +42,35 @@ class GemmaLlmService implements LlmService {
@override
Future<void> load() async {
throw UnimplementedError(
'GemmaLlmService.load: pending OQ-1 (model URL + flutter_gemma).',
);
if (_loaded) return;
if (!await File(modelPath).exists()) {
throw FileSystemException('model file missing', modelPath);
}
if (!_initialized) {
await FlutterGemma.initialize(huggingFaceToken: _hfToken);
_initialized = true;
}
await FlutterGemma.installModel(
modelType: ModelType.gemma4,
fileType: ModelFileType.litertlm,
).fromFile(modelPath).install();
final model = await FlutterGemma.getActiveModel(maxTokens: 2048);
_model = model;
_loaded = true;
}
@override
Future<void> unload() async {
final m = _model;
_model = null;
_loaded = false;
if (m != null) {
try {
await m.close();
} catch (_) {
// Best-effort cleanup — runtime may already be torn down.
}
}
}
@override
@@ -38,8 +78,90 @@ class GemmaLlmService implements LlmService {
String prompt,
Map<String, dynamic> schema,
) async {
throw UnimplementedError(
'GemmaLlmService.generateStructured: pending OQ-1.',
if (!_loaded || _model == null) {
throw StateError('LlmService not loaded');
}
final fnName = schema['name'];
final fnParams = schema['parameters'];
if (fnName is! String || fnName.isEmpty) {
throw ArgumentError('schema.name missing');
}
if (fnParams is! Map) {
throw ArgumentError('schema.parameters missing');
}
final fnDesc = (schema['description'] as String?) ?? '';
final tool = Tool(
name: fnName,
description: fnDesc,
parameters: Map<String, dynamic>.from(fnParams),
);
final chat = await _model!.createChat(
modelType: ModelType.gemma4,
supportsFunctionCalls: true,
toolChoice: ToolChoice.required,
tools: [tool],
);
try {
await chat.addQueryChunk(Message.text(text: prompt, isUser: true));
final stream = chat.generateChatResponseAsync();
return await collectFunctionCall(stream, fnName);
} finally {
try {
await chat.close();
} catch (_) {
// Native session close failure is non-fatal — log + continue.
}
}
}
}
/// Extracts the first `FunctionCallResponse(name == expectedName)` from
/// a flutter_gemma response stream. `TextResponse` / `ThinkingResponse`
/// events are skipped. A mismatched name throws fast.
///
/// File-private under `_collectFunctionCall` from [GemmaLlmService];
/// exposed as a top-level via `@visibleForTesting` so unit tests can
/// feed synthetic streams (see fn-spec §D, 8 test cases).
@visibleForTesting
Future<Map<String, dynamic>> collectFunctionCall(
Stream<ModelResponse> stream,
String expectedName,
) async {
Map<String, dynamic>? result;
String? wrongName;
try {
await for (final event in stream) {
if (event is FunctionCallResponse) {
if (event.name == expectedName) {
result = Map<String, dynamic>.from(event.args);
break;
} else {
wrongName = event.name;
break;
}
}
if (event is ParallelFunctionCallResponse && event.calls.isNotEmpty) {
final first = event.calls.first;
if (first.name == expectedName) {
result = Map<String, dynamic>.from(first.args);
} else {
wrongName = first.name;
}
break;
}
// TextResponse / ThinkingResponse: skip.
}
} catch (_) {
// Discard raw error to avoid leaking prompt content in logs/crash
// reports — the caller surfaces a generic message.
throw const FormatException('stream error');
}
if (wrongName != null) {
throw FormatException('unexpected function: $wrongName');
}
if (result == null) {
throw const FormatException('no function call emitted');
}
return result;
}

41
app/lib/data/ai/model_lifecycle.dart
View File

@@ -233,21 +233,37 @@ class ModelLifecycle {
/// opt-out: delete model file + clear all ai_* meta keys (except opt_in
/// which the caller toggles). Returns freed bytes (0 if nothing existed).
/// Idempotent.
///
/// F2 hardening (#218): per-file try/catch so a single OS-level delete
/// failure (locked file, permission flake) does not abort the whole
/// purge — meta keys still get cleared and the orphan file becomes a
/// background storage concern rather than a stuck "opt-out failed"
/// state. The freed-bytes count only reflects successful deletes.
Future<int> purge() async {
int freed = 0;
final pathStr = await meta.find(AiMetaKeys.modelPath);
if (pathStr != null) {
final f = File(pathStr);
if (f.existsSync()) {
freed += await f.length();
await f.delete();
try {
final f = File(pathStr);
if (f.existsSync()) {
final size = await f.length();
await f.delete();
freed += size;
}
} catch (_) {
// Best-effort; leave orphan file, continue purging meta.
}
}
final tempPath = '${await _modelPath()}.tmp';
final temp = File(tempPath);
if (temp.existsSync()) {
freed += await temp.length();
await temp.delete();
try {
final tempPath = '${await _modelPath()}.tmp';
final temp = File(tempPath);
if (temp.existsSync()) {
final size = await temp.length();
await temp.delete();
freed += size;
}
} catch (_) {
// Same as above — best-effort cleanup of the .tmp partial.
}
for (final k in [
AiMetaKeys.modelPath,
@@ -255,7 +271,12 @@ class ModelLifecycle {
AiMetaKeys.downloadState,
AiMetaKeys.downloadBytes,
]) {
await meta.remove(k);
try {
await meta.remove(k);
} catch (_) {
// Meta is a single sqlite table; failures here are rare.
// Swallow so the loop completes even if one key errors.
}
}
return freed;
}

58
app/lib/main.dart
View File

@@ -1,7 +1,10 @@
import 'package:flutter/material.dart';
import 'package:flutter_riverpod/flutter_riverpod.dart';
import 'data/ai/gemma_llm_service.dart';
import 'data/ai/llm_service.dart';
import 'data/ai/model_lifecycle.dart';
import 'data/db/daos/meta_dao.dart';
import 'state/ai_providers.dart';
import 'state/providers.dart';
import 'ui/screens/habit_list_screen.dart';
@@ -12,15 +15,62 @@ Future<void> main() async {
runApp(ProviderScope(
overrides: [
appDatabaseProvider.overrideWithValue(db),
// OQ-1 pending: production-ready GemmaLlmService is wired here once
// model URL + SHA are pinned and flutter_gemma is added. Until then,
// MockLlmService keeps the app graceful (suggestFrame returns []).
llmServiceProvider.overrideWithValue(MockLlmService()),
// #218: real GemmaLlmService when model file is on disk + verified,
// MockLlmService otherwise. The provider is read lazily by the frame
// suggestion flow, so the resolution is dynamic per call.
llmServiceProvider.overrideWith((ref) {
return _LazyLlmService(
lifecycle: ref.watch(modelLifecycleProvider),
meta: ref.watch(metaDaoProvider),
);
}),
],
child: const LifeHelperApp(),
));
}
/// Adapter that lazily resolves between [GemmaLlmService] (when the
/// model file exists + meta is intact) and [MockLlmService] (fallback,
/// graceful empty candidates). Keeps the rest of the app unaware of
/// the difference — `suggestFrame` only sees [LlmService].
class _LazyLlmService implements LlmService {
_LazyLlmService({required this.lifecycle, required this.meta});
final ModelLifecycle lifecycle;
final MetaDao meta;
LlmService? _delegate;
Future<LlmService> _resolve() async {
if (_delegate != null) return _delegate!;
final avail = await lifecycle.checkAvailability();
final path = await meta.find(AiMetaKeys.modelPath);
if (avail == ModelAvailability.ready && path != null) {
_delegate = GemmaLlmService(modelPath: path);
} else {
_delegate = MockLlmService();
}
return _delegate!;
}
@override
bool get isLoaded => _delegate?.isLoaded ?? false;
@override
Future<void> load() async => (await _resolve()).load();
@override
Future<void> unload() async {
final d = _delegate;
if (d != null) await d.unload();
}
@override
Future<Map<String, dynamic>> generateStructured(
String prompt,
Map<String, dynamic> schema,
) async =>
(await _resolve()).generateStructured(prompt, schema);
}
class LifeHelperApp extends StatelessWidget {
const LifeHelperApp({super.key});

21
app/lib/state/ai_providers.dart
View File

@@ -10,19 +10,24 @@ import '../domain/ai/suggest_frame.dart';
import '../domain/models/frame_pattern.dart';
import 'providers.dart';
/// Default config for the on-device Gemma model (#215).
/// OQ-1: URL + SHA-256 pinned in Developer phase. Until then, downloads are
/// disabled (AI toggle is gated behind these constants being real).
const _kModelUrlPlaceholder =
'https://example.invalid/gemma4-e2b-q4.bin'; // OQ-1
const _kModelShaPlaceholder = 'PENDING_OQ_1';
/// Gemma 4 E2B instruction-tuned LiteRT-LM checkpoint (#218 OQ-1 resolved).
/// Hosted on HuggingFace `litert-community/gemma-4-E2B-it-litert-lm`.
/// File ≈ 2.41GB; SHA-256 pinned for integrity check.
///
/// Tests / placeholder builds may override `modelLifecycleProvider` with
/// fixture URLs. Production builds optionally inject a private mirror via
/// `--dart-define=GEMMA_MODEL_URL=...` (see main.dart).
const _kModelUrl =
'https://huggingface.co/litert-community/gemma-4-E2B-it-litert-lm/resolve/main/gemma-4-E2B-it.litertlm';
const _kModelSha256 =
'181938105e0eefd105961417e8da75903eacda102c4fce9ce90f50b97139a63c';
final modelLifecycleProvider = Provider<ModelLifecycle>((ref) {
return ModelLifecycle(
meta: ref.watch(metaDaoProvider),
config: ModelConfig(
url: Uri.parse(_kModelUrlPlaceholder),
expectedSha256: _kModelShaPlaceholder,
url: Uri.parse(_kModelUrl),
expectedSha256: _kModelSha256,
),
);
});

7
app/lib/ui/screens/settings_screen.dart
View File

@@ -51,7 +51,7 @@ class _AiSection extends ConsumerWidget {
SwitchListTile(
title: const Text('AI 도움 켜기'),
subtitle: const Text(
'Gemma 4 E2B 모델 ≈ 1.5GB. 모든 처리는 단말에서 일어납니다.',
'Gemma 4 E2B 모델 ≈ 2.4GB. 모든 처리는 단말에서 일어납니다.',
),
value: optIn,
onChanged: (v) async {
@@ -125,9 +125,10 @@ class _AiSection extends ConsumerWidget {
style: TextStyle(fontWeight: FontWeight.w600),
),
SizedBox(height: 12),
_Bullet('파일 크기: 약 1.5GB'),
_Bullet('파일 크기: 약 2.4GB'),
_Bullet('WiFi 연결을 권장합니다'),
_Bullet('모든 처리는 단말에서만 일어나며, 입력 텍스트는 외부로 전송되지 않습니다'),
_Bullet('Gemma 이용약관(ai.google.dev/gemma/terms)에 동의합니다'),
],
),
actions: [
@@ -156,7 +157,7 @@ class _AiSection extends ConsumerWidget {
Text('모델 파일이 단말에서 삭제됩니다.'),
SizedBox(height: 8),
Text(
'약 1.5GB 의 저장공간이 확보돼요.',
'약 2.4GB 의 저장공간이 확보돼요.',
style: TextStyle(fontSize: 13, color: Colors.grey),
),
SizedBox(height: 4),