Documentation

System Requirements

QUAILS is a local web application. It runs a small Python backend and a React frontend on your machine. You interact with it through your browser.

Installation

QUAILS consists of a Python backend and a React frontend. Both need to be running at the same time.

1. Install the backend

cd backend
pip install -r requirements.txt
python app.py

The backend starts on http://localhost:5174. Verify it's running before starting the frontend.

2. Install and run the frontend

cd frontend
npm install
npm run dev

The frontend starts on http://localhost:5173 and automatically proxies API calls to the backend. Open this URL in your browser.

One-click launchers

Platform-specific launcher scripts are included in the project root:

• start.bat — Windows
• start.command — macOS (double-click in Finder)
• start.sh / start_linux.sh — Linux

These scripts validate your Python version, start both backend and frontend, and open the browser automatically.

Quick Start Guide

The fastest path from install to a real analysis run:

Creating a Project

Each project in QUAILS is a self-contained folder that stores your documents, analysis pipeline, results, and git history. Projects live in the projects/ directory inside the QUAILS installation.

Project templates

When creating a new project, choose from two templates:

• Basic Thematic Analysis — pre-configured pipeline with an Open-Coding Analysis block, Theme Identification, Category Builder, and two Human Review checkpoints. The recommended starting point for most qualitative projects.
• Blank Project — an empty canvas. Start from scratch and add only the blocks you need.

Uploading Documents

Click Add Documents in the left sidebar of any project workspace. You can upload files individually or as a named folder collection.

Folder collections

Check "Upload as a folder collection" to group multiple files under a shared name. This lets you scope individual analysis blocks to specific collections — for example, analyzing interview transcripts separately from field notes.

Supported File Formats

Units of Analysis

Every analysis block has an independent unit type setting. QUAILS parses all documents into a four-level hierarchy — Document → Section → Paragraph → Sentence — and you choose at which level to analyze.

The Flow Canvas

The Analysis Builder tab is a visual drag-and-drop canvas where you design your analysis pipeline. Blocks represent processing steps; edges between them define the flow of data.

localhost:5173 — Analysis Builder

Screenshot: Flow canvas with a complete multi-block pipeline — Project Spec → Open-Coding Analysis → Human Review → Theme Identification → Category Builder → Outputs & Report

Canvas controls

• Scroll to zoom · Click and drag to pan
• Click a block to open its settings panel on the right
• Drag from a block's output handle to another block's input to connect them
• Use the Step Order panel (top-right) to reorder parallel blocks
• Use the Validate button to check for configuration errors before running

Block Types Overview

Connecting & Ordering Blocks

Drag from the bottom handle of one block to the top handle of the next to connect them. QUAILS automatically wires the upstream block's output CSV as the input to the downstream block.

When two analysis blocks are connected directly, a Human Review node is automatically inserted between them as a floating checkpoint. This is by design — you are expected to review AI output before it flows downstream.

Step ordering

The Step Order panel (top-right of the canvas) shows all analysis blocks in their current sequence. Use the ▲/▼ arrows to reorder parallel blocks — blocks that have a dependency (one feeds the other) cannot be swapped. The step badge on each canvas node updates live.

Project Spec Node

The Project Spec is the start node of every pipeline. It controls the LLM sampling parameters that apply to all analysis blocks in the pipeline.

Open-Coding Analysis

The Open-Coding block applies a two-step LLM process to every text unit in your corpus:

1. Detection — asks the AI whether the unit is relevant to your research question (TRUE or FALSE), with a confidence score
2. Labeling — if detected as TRUE above your confidence threshold, asks the AI to assign a descriptive free-text label

Both steps are followed by a structured reasoning follow-up that records why the AI made its decision. All results — label, confidence, reasoning — are written to a CSV for every unit.

Configuration fields

• Detection Prompt — the question the AI answers TRUE/FALSE. Example: "Does this text contain a discussion of institutional barriers to participation?"
• Label Prompt — how the AI should label TRUE units. Example: "Assign a single descriptive code that captures the main theme."
• Detection Threshold — minimum confidence (0–100%) to treat a detection as TRUE. Units below this threshold go to the "false" CSV.
• Label Threshold — minimum confidence for the assigned label to be accepted.
• Unit Type — sentence, paragraph, section, document, or line
• Data Source — all documents, a single document, or a named folder collection
• Context Window — number of neighboring units (0–10) to include as context around each unit

Rubric-Based Analysis

The Rubric block scores every text unit against a set of researcher-defined rubric items in a single pass. There is no detection step — every unit is always scored on every rubric item.

For each rubric item you define:

• Title — short name for the rubric item (e.g., "Emotional tone")
• Description — what this dimension captures
• Scoring Criteria — the specific instruction given to the AI for scoring

The AI scores the unit, reports a confidence level, and provides a plain-language justification. If confidence falls below the label threshold, the score is left blank in the output — but confidence and justification are always recorded.

localhost:5173 — Rubric Block Settings

Screenshot: Rubric block settings panel — showing three rubric item cards, each with title, description, and scoring criteria fields, plus the label threshold slider

Theme Identification

The Theme Identification block implements a constant comparison method. It reads labeled units from one or more upstream analysis blocks and builds a growing list of themes.

For each unit, the AI:

1. Compares the unit's label against all existing themes
2. If a match is found (TRUE), assigns that theme
3. If no match (FALSE), generates a new theme title and adds it to the list

This block accepts multiple upstream inputs — you can feed labels from both an Open-Coding block and a Rubric block into the same Theme Identification step.

Category Builder

The Category Builder takes the full list of identified themes and makes a single LLM call to organize them into a user-defined number of top-level categories.

Set the Maximum Categories field (default: 6) in the block settings. The AI returns a nested structure mapping each theme to a category, which is saved as a Markdown report.

Custom Prompt

The Custom Prompt block is a free-form LLM step. You write the system prompt and user message template yourself, and direct it at any input data or upstream block output.

Use it for tasks that don't fit the standard block types: sentiment classification, entity extraction, translation, summarization, or any other text transformation.

Human Review Node

Human Review nodes are automatically inserted between connected analysis blocks. They serve as gated checkpoints — a visual reminder that a human should inspect the AI's output before the next analysis step runs.

The Human Review node's settings panel shows all output files from the upstream block, with direct links to open them in the Data Browser or file explorer. Mark the review complete when you're satisfied with the output.

Setting Up Ollama

Ollama is a background application that runs LLM models locally on your computer. Download it from ollama.com and install it before starting QUAILS.

Downloading a model

You need at least one model downloaded before you can run analysis. In QUAILS's AI Settings page, click "Test my system" in the Ollama card. QUAILS will scan your hardware and present a ranked list of compatible models. Click "Download" on any model to pull it via Ollama.

Alternatively, download models from the command line:

ollama pull gemma3
ollama pull llama3.2
ollama pull mistral

Selecting a model in QUAILS

1. Go to AI Settings (gear icon on the project list)
2. Click "Load my installed models" in the Ollama card
3. Select a model from the dropdown
4. Click "Set as Active"

Cloud Providers & API Keys

QUAILS supports OpenAI, Anthropic (Claude), Google Gemini, and Azure OpenAI as cloud providers. To use any of them:

1. Obtain an API key from the provider's website
2. In AI Settings, find the provider card and paste your key into the API Key field
3. Click "Save Key" — the key is stored locally in backend/.env (never transmitted to the developer)
4. Click "Load my installed models" to fetch available models
5. Select a model and click "Set as Active"

Context Window

The context window is the maximum number of tokens Ollama can hold in memory for a single request. The default is 2,048 tokens — often too small for longer text units.

Adjust it using the slider in the Ollama provider card on the AI Settings page:

• 24,000 tokens — recommended minimum for sentence-level analysis
• 65,536 tokens — for paragraph- or section-level analysis
• 131,072 tokens — for very long documents or large context windows

Hardware Recommendations

QUAILS integrates llmfit, an open-source tool that scans your CPU, RAM, GPU, and VRAM and ranks compatible Ollama models by suitability for your hardware.

Each model is scored across four dimensions:

• Fit (35%) — does it run within your hardware constraints?
• Speed (35%) — how fast will inference be?
• Quality (20%) — model capability rating
• Context (10%) — maximum context window the model supports

Click "Test my system" in the Ollama card to run a scan. llmfit is automatically downloaded the first time you use it — no manual installation required.

Why Calibrate First

Alignment testing is the most important step in any QUAILS workflow. Before running full analysis on your corpus, you need to verify that the AI's labels agree with your own judgement — and that requires measuring it.

The recommended workflow is: design prompts → generate sample → human label → compare → refine → repeat → run full analysis. Skipping calibration means you have no evidence that the AI is applying your coding schema the way you intend.

Generating a Sample

1. Go to the Alignment tab in your project workspace
2. Select a block from the dropdown
3. Set the sample size (default: 20 units)
4. Click "Generate Sample" — QUAILS randomly selects units from the block's configured data source

The same sample is used for both human labeling and LLM comparison, ensuring a fair apples-to-apples comparison.

Human Labeling

After generating a sample, the Human Labeling panel shows each unit numbered and scrollable. For each unit:

• Open-Coding blocks: select TRUE or FALSE from the detection dropdown, then type your label if TRUE
• Rubric blocks: enter a score for each rubric item

Use the Save button on each row to save individual labels as you go — your work is preserved even if you navigate away. The scrollbar is intentionally prominent (indigo, 14px) to encourage reviewing all units.

Running Comparisons

Once all sample units are labeled, click "Run LLM Comparison." QUAILS runs the block's configured prompts on the same sample and computes alignment metrics:

• Cohen's Kappa — detection agreement (TRUE/FALSE)
• Label Similarity — TF-IDF cosine + Jaccard overlap for free-text label comparison
• Rubric Metrics — per-item score comparison for rubric blocks
• Overall Alignment Score — weighted blend of all metrics

Results appear in the Comparison Results panel with a per-unit breakdown table showing your label vs. the AI's label side by side.

Interpreting Cohen's Kappa

Cohen's Kappa (κ) measures how much better than chance your agreement with the AI is, correcting for the probability of random agreement.

Prompt Templates

The Default Prompt Editor tab exposes every LLM prompt and activity log template used by QUAILS. You can customize them at the project level (overrides for this project only) or at the global level (applies to all new projects).

Each block type has templates for:

• System Template — the system role message wrapping your prompt
• Unit Introduction — preamble prepended before the unit text
• Context Templates — how pre- and post-context is framed when context window > 0
• Reasoning Prompt — the follow-up question asking the AI to explain its decision
• Activity Log — what gets printed in the live log panel for each unit

Use the "Preview full prompt structure" collapsible at the bottom of each task page to see the exact message sequence the AI will receive before you run anything.

Confidence Scoring Explained

QUAILS measures AI confidence using token log-probabilities (logprobs) — a number that reflects how strongly the model "committed" to generating each token in its response.

The formula: confidence = e^logprob × 100

For multi-token responses (e.g. a two-word label), QUAILS uses the geometric mean of all token logprobs. The result is a 0–100% confidence score.

Export & Import Projects

Projects can be exported as .qpkg files — a ZIP archive containing your project configuration, flow pipeline, and optionally all data files.

Exporting

Click the Export button on any project card on the project list. Options:

• Include data — include corpus files, parsed documents, analysis results, and alignment tests
• Encrypt with password — AES-256 encryption for sensitive data

Importing

Click "Import Project" on the project list, select a .qpkg or .zip file, and enter the password if required. QUAILS generates a fresh project ID and rewrites all internal unit IDs to avoid conflicts.

Version Control

Every QUAILS project has its own git repository, managed automatically using GitPython. After each completed analysis block, QUAILS commits the updated project files with a descriptive message.

This gives you a complete, timestamped history of every change to your analysis configuration and results. You can inspect a project's history using standard git tools from the project's folder inside projects/.