github · prankster009 · Jul 22, 2026
@@ -18,6 +18,9 @@ JS:
   - any: [ 'javascript/**/*' ]
   - change-notes/**/*javascript*
 
+Lua:
+  - lua/**/*
+
 Kotlin:
   - java/kotlin-extractor/**/*
   - java/ql/test-kotlin*/**/*

@@ -31,4 +31,4 @@ jobs:
       - name: Fail if there are any errors with existing change notes
 
         run: |
-          codeql pack release --groups actions,cpp,csharp,go,java,javascript,python,ruby,shared,swift -examples,-test,-experimental
+          codeql pack release --groups actions,cpp,csharp,go,java,javascript,lua,python,ruby,shared,swift -examples,-test,-experimental
@@ -0,0 +1,206 @@
+# Lua 5.1 language analysis for CodeQL
+
+This directory adds Lua 5.1 as a new language analysis capability to CodeQL.
+It provides a self-contained extractor, database schema, QL libraries, query
+packs, security queries, tests, and documentation for compiled Lua 5.1
+bytecode. Existing `.luac` files can be analyzed directly. Lua source is
+analyzed by compiling it with `luac5.1` while preserving the source-tree
+layout, then analyzing the generated bytecode.
+
+Plain `.lua` files are recorded as source inventory. Semantic, data-flow, and
+taint facts come from `.luac` files; this implementation does not claim a Lua
+source AST.
+
+## Capability matrix
+
+| Capability | Support |
+| --- | --- |
+| Language version | Lua 5.1 bytecode. Other bytecode versions are rejected with diagnostics. |
+| Source workflow | Compile `.lua` files with `luac5.1`, preserving relative paths, then analyze the resulting `.luac` tree. |
+| Bytecode input | Direct recursive ingestion of `.luac` files. |
+| Bytecode model | Artifact profiles, prototypes, instructions, constants, closures, upvalues, register events, and call sites. |
+| Local semantics | Control flow, dominance, reaching definitions, def-use, globals, upvalues, tables, and overwrite behavior. |
+| Calls and modules | Local call targets, literal `require`, module resolution, returned exports, and module-field calls. |
+| Interprocedural flow | Arguments, varargs, returns, open results, tail calls, and callsite-balanced summaries. |
+| Taint flow | Local and cross-module source-to-sink paths over the bytecode value graph. |
+| Sources and sinks | Resolved API-name models for external inputs and command-execution calls. |
+| Sanitizers | Resolved API-name models with path-local and value-local classification. |
+| Security output | `lua/command-injection` path findings in the experimental Lua security suite. |
+| Review output | Sanitized paths and detailed bytecode/model facts as directly runnable diagnostic queries. |
+| Diagnostics | Explicit rejection facts for malformed chunks and unsupported Lua versions or profiles. |
+| Result formats | BQRS, CSV or JSON decoding, and SARIF for path queries. |
+
+## Prerequisites
+
+- A CodeQL CLI distribution.
+- This repository checkout, used as the Lua extractor and pack search root.
+- `luac5.1` only when starting from Lua source.
+
+The public repository contains the extractor and packs. Until Lua is included
+in an official CodeQL CLI bundle, commands must include this checkout in
+`--search-path`. CodeQL Action and official bundle registration are separate
+upstream product integration work.
+
+The examples below run from the repository root:
+
+```bash
+CODEQL=/absolute/path/to/codeql
+ROOT=$(pwd)
+```
+
+## Analyze existing bytecode
+
+Set `INPUT` to an absolute directory containing `.luac` files. The directory
+hierarchy below `INPUT` becomes the stable module layout in the database.
+
+```bash
+INPUT=/absolute/path/to/lua-bytecode
+DB=/tmp/codeql-lua51-db
+
+env -u PYTHONPATH CODEQL_LUA_BYTECODE_INPUT_ROOT="$INPUT" \
+  "$CODEQL" database create "$DB" \
+    --overwrite \
+    --language=lua \
+    --source-root="$INPUT" \
+    --build-mode=none \
+    --search-path="$ROOT:$ROOT/lua"
+```
+
+Run the experimental security suite and write SARIF:
+
+```bash
+"$CODEQL" database analyze "$DB" \
+  "$ROOT/lua/ql/src/codeql-suites/lua-security-experimental.qls" \
+  --search-path="$ROOT:$ROOT/lua" \
+  --format=sarifv2.1.0 \
+  --output=/tmp/lua-results.sarif
+```
+
+## Analyze source
+
+Compile each source file into a separate output tree with the same relative
+path. Use a new or empty bytecode directory so stale output cannot enter the
+database.
+
+```bash
+set -euo pipefail
+
+SOURCE_ROOT=/absolute/path/to/lua-source
+BYTECODE_ROOT=$(mktemp -d)
+
+find "$SOURCE_ROOT" -type f -name '*.lua' -print0 |
+while IFS= read -r -d '' source; do
+  relative=${source#"$SOURCE_ROOT"/}
+  output="$BYTECODE_ROOT/${relative%.lua}.luac"
+  mkdir -p "$(dirname "$output")"
+  luac5.1 -o "$output" "$source"
+done
+```
+
+Create the database from the compiled tree. The original source tree remains
+the compilation input; semantic analysis and database module paths come from
+the matching bytecode layout.
+
+```bash
+DB=/tmp/codeql-lua51-source-db
+
+env -u PYTHONPATH CODEQL_LUA_BYTECODE_INPUT_ROOT="$BYTECODE_ROOT" \
+  "$CODEQL" database create "$DB" \
+    --overwrite \
+    --language=lua \
+    --source-root="$BYTECODE_ROOT" \
+    --build-mode=none \
+    --search-path="$ROOT:$ROOT/lua"
+```
+
+Use the same `database analyze` command from the bytecode workflow.
+
+Creating a database directly from a `.lua` source tree without
+`CODEQL_LUA_BYTECODE_INPUT_ROOT` records source-file inventory only. It does
+not produce source AST, semantic, data-flow, or taint facts.
+
+## Query interfaces
+
+The Lua 5.1 analysis capability is mature. Its user-facing queries follow
+CodeQL's experimental support lifecycle for initial upstream adoption; this
+publication status does not apply to the extractor, schema, or analysis
+libraries. The default supported suites intentionally select no Lua query yet.
+
+| Query | Purpose | Suite status |
+| --- | --- | --- |
+| [`lua/command-injection`](ql/src/experimental/Security/CWE-078/CommandInjection.ql) | Unsanitized external-data flow to command execution. | Experimental security suite and direct execution. |
+| [`lua/diagnostics/sanitized-command-flow`](ql/src/experimental/Diagnostics/SanitizedCommandFlow.ql) | Review paths suppressed by a sanitizer on the relevant route. | Experimental direct/manual diagnostic query. |
+| [`lua/diagnostics/bytecode-facts`](ql/src/experimental/Diagnostics/LuaBytecodeFacts.ql) | Detailed tagged bytecode, flow, module, and rule facts. | Experimental direct/manual table query. |
+
+The active and sanitized queries are path problems with `file`, `source`,
+`sink`, and message output plus native path edges. Their formal query help is
+next to each query. The facts query returns a tagged union suitable for CSV or
+JSON consumers. See [`ql/src/README.md`](ql/src/README.md) for output schemas.
+
+Run a direct query and decode it as CSV:
+
+```bash
+QUERY="$ROOT/lua/ql/src/experimental/Security/CWE-078/CommandInjection.ql"
+
+"$CODEQL" query run "$QUERY" \
+  --database="$DB" \
+  --search-path="$ROOT:$ROOT/lua" \
+  --output=/tmp/lua-results.bqrs
+
+"$CODEQL" bqrs decode /tmp/lua-results.bqrs \
+  --format=csv \
+  --output=/tmp/lua-results.csv
+```
+
+## Complex multi-file example
+
+The committed integration corpus contains 46 source files and 46 matching Lua
+5.1 bytecode files in a realistic module layout. Its compact oracle proves
+46 accepted bytecode artifacts, zero diagnostics, nine active findings, zero
+sanitized findings, and module, interprocedural, and cross-module behavior.
+
+- [Corpus manifest](ql/test/library-tests/complex-bytecode-integration/SAMPLE-MANIFEST.md)
+- [Compact oracle](ql/test/library-tests/complex-bytecode-integration/CorpusAcceptance.expected)
+
+Reproduce this example from the repository root:
+
+```bash
+"$CODEQL" test run \
+  lua/ql/test/library-tests/complex-bytecode-integration \
+  --search-path="$ROOT:$ROOT/lua"
+```
+
+## Validation
+
+Run the complete extractor and QL regression suites:
+
+```bash
+python3 -m unittest discover -s lua/tools/tests -p 'test_*.py'
+
+"$CODEQL" test run lua/ql/test \
+  --search-path="$ROOT:$ROOT/lua"
+
+"$CODEQL" query compile --check-only --warnings=error \
+  --search-path="$ROOT:$ROOT/lua" \
+  lua/ql/src/experimental/Security/CWE-078/CommandInjection.ql \
+  lua/ql/src/experimental/Diagnostics/SanitizedCommandFlow.ql \
+  lua/ql/src/experimental/Diagnostics/LuaBytecodeFacts.ql
+
+cmp lua/lua.dbscheme lua/ql/lib/lua.dbscheme
+cmp lua/lua.dbscheme.stats lua/ql/lib/lua.dbscheme.stats
+```
+
+The [test matrix](ql/test/library-tests/README.md) lists the stable behavioral
+boundaries. Generated databases, reports, BQRS, SARIF, logs, and profiles are
+not committed.
+
+## Input boundary
+
+The bytecode loader accepts Lua 5.1 chunks with format version 0, either
+endianness, positive C `int` and `size_t` widths, 4-byte instructions, and
+8-byte floating-point `lua_Number` values. Malformed chunks and unsupported
+profiles emit diagnostics instead of partial semantic facts.
+
+The implementation is self-contained in this repository and does not import
+precomputed findings. Production analysis derives results from extracted
+bytecode facts and the QL libraries under `lua/ql/lib/codeql/lua`.
@@ -0,0 +1,74 @@
+# Lua 5.1 bytecode schema
+
+`lua/lua.dbscheme` is the extractor schema authority. The mirrored schema and
+statistics files under `lua/ql/lib` let the QL packs resolve the same relations
+and must remain byte-identical to their extractor-package counterparts:
+
+```bash
+cmp lua/lua.dbscheme lua/ql/lib/lua.dbscheme
+cmp lua/lua.dbscheme.stats lua/ql/lib/lua.dbscheme.stats
+```
+
+## Stable identities
+
+- Artifact and module paths are normalized, source-root-relative paths.
+- Prototype IDs use `root` followed by ordinal child components such as
+  `root.2.1`.
+- Instruction sites use `<prototype>@pc<pc>`.
+- Register values use `<prototype>@pc<pc>:r<slot>`.
+- Callsite IDs use the instruction-site form and are always interpreted with
+  caller module and caller prototype ownership.
+- Missing debug names remain explicit as unavailable metadata; they are not
+  guessed from paths or benchmark results.
+
+These identities describe input evidence. Production rules derive behavior
+from extracted relations and resolved API semantics rather than input
+identities.
+
+## Relation families
+
+| Family | Relations | Contract |
+| --- | --- | --- |
+| Files and locations | `files`, `folders`, `containerparent`, `locations_default`, `sourceLocationPrefix` | Minimal entities required for CodeQL result locations. |
+| Input inventory | `lua_source_files`, `lua_artifacts`, `lua_profiles`, `lua_diagnostics` | Accepted bytecode and rejected profiles are disjoint. Diagnostic artifacts do not emit partial semantic success facts. |
+| Bytecode model | `lua_prototypes`, `lua_instructions`, `lua_constants`, `lua_register_events`, `lua_semantic_steps`, `lua_closure_values`, `lua_call_sites`, `lua_upvalues` | Typed Lua 5.1 structure and register effects. |
+| Local semantics | `lua_local_flows`, `lua_control_flow_edges`, `lua_dominator_tree_intervals`, `lua_analysis_boundaries` | Reaching definitions, control flow, dominance, and explicit unsupported boundaries. |
+| State carriers | `lua_table_field_flows`, `lua_global_flows`, `lua_upvalue_flows` | Static table fields may be precise; dynamic keys never create precise field flow. Same-table conservative object flow remains available. |
+| Resolution | `lua_call_resolutions`, `lua_literal_requires`, `lua_module_resolutions`, `lua_module_exports` | Resolved, ambiguous, and unresolved states remain explicit. No guessed callee or module target is emitted. |
+| Interprocedural flow | `lua_interprocedural_flows` | Argument, vararg, return, open-result, and tailcall rows retain caller module, caller prototype, callsite, callee module, and callee prototype ownership. |
+| Mapping evidence | `lua_mapping_markers`, `lua_mapping_marker_diagnostics` | Source/bytecode mapping states; not a source AST contract. |
+
+## Analysis invariants
+
+1. An input is either accepted or diagnostic; malformed input cannot contribute
+   ordinary bytecode, flow, or report facts.
+2. Local reaching definitions reject overwritten values while preserving
+   branch and loop alternatives.
+3. Static table fields may use field-sensitive flow. Unknown or dynamic keys
+   use conservative table-object flow and do not acquire a fabricated field
+   name.
+4. Interprocedural edges are callsite-balanced. A return from one callsite
+   cannot satisfy another callsite's route.
+5. Fixed and producer-proven open arguments/results are represented
+   structurally. Missing producer evidence is an explicit boundary, not a
+   guessed slot range.
+6. Sanitizer and report semantics are derived in QL from typed calls and the
+   generic flow graph. Extractor relations do not contain final benchmark
+   findings.
+
+## Changing the schema
+
+When a relation changes:
+
+1. Update both dbscheme copies and both statistics copies identically.
+2. Update the TRAP producer in `lua/tools/index_lua_files.py`.
+3. Update the typed QL wrapper in `lua/ql/lib/codeql/lua/`.
+4. Add or update a committed positive and negative test at the complete
+   behavioral boundary.
+5. Run the Python, QL, query-compile, and schema-mirror commands in
+   [`README.md`](README.md).
+
+The schema and QL APIs support the repository-local Lua 5.1 bytecode analysis
+implementation. Official bundle and CodeQL Action registration are separate
+upstream product integration work. Source parsing and non-Lua-5.1 bytecode are
+not claimed.
@@ -0,0 +1,22 @@
+name: "lua"
+display_name: "Lua"
+version: 0.1.0
+column_kind: "utf8"
+legacy_qltest_extraction: true
+build_modes:
+  - none
+default_queries:
+  - codeql/lua-queries
+github_api_languages:
+  - Lua
+scc_languages:
+  - Lua
+file_types:
+  - name: lua
+    display_name: Lua source files
+    extensions:
+      - .lua
+  - name: lua-bytecode
+    display_name: Lua 5.1 bytecode files
+    extensions:
+      - .luac