After processing a batch of smaps files, you can use this script to analyze the entire run. It reads a directory of processed smaps files, tracks memory usage over time, and generates a summary report.

The report includes:

* The top 10 largest memory consumers by PSS and RSS at the end of the run.

* The top 10 memory regions that have grown the most in PSS and RSS over the duration of the run.

* The overall change in total PSS and RSS.

Optional: The path to a file where the JSON analysis output will be saved.

1. **Print a text-based analysis to the console:**

To simplify the analysis process, the `run_analysis_pipeline.py` script combines the batch processing, analysis, and visualization steps into a single command.

This script takes a directory of raw smaps logs and generates the final visualization PNG, handling all intermediate steps automatically.

* `<RAW_LOG_DIR>` (required positional argument)

The path to the directory containing the raw smaps log files.

* `--output_image` (type: `str`, default: `smaps_analysis.png`)

The path where the final output PNG image will be saved.

* `--platform` (type: `str`, choices: `android`, `linux`, default: `android`)

Specify the platform for platform-specific aggregations.

The accuracy of this toolchain depends on its aggregation rules, which are heuristics based on known memory patterns. As Cobalt, Android, and third-party libraries evolve, new memory region names can appear. It is crucial to periodically check for and categorize these new regions to prevent gaps in the analysis.

1. **Temporarily Disable Aggregation:** Open `run_analysis_pipeline.py` and remove the `-d` (or `--aggregate_android`) flag from the `batch_args` list. This will cause the batch processor to output a "raw" report with no special grouping.

2. **Run the Pipeline:** Execute the modified script on a recent and representative set of `smaps` logs.

```bash

3. **Examine the Raw Output:** The analysis printed to the console will now be much more detailed. Scan the "Top Largest Consumers" and "Top Memory Increases" lists. Look for patterns or repeated names that are not being grouped, such as:

* New `[anon:<name>]` labels (e.g., we discovered `[anon:scudo:*]`).

* Driver or shared memory regions (e.g., `/dev/ashmem/*`).

* JIT or code cache regions (e.g., `/memfd:jit-cache`).

* Any other large, unexplained region that appears frequently.

4. **Add New Aggregation Rules:** Open `read_smaps.py` and add new `re.sub()` rules within the `if args.aggregate_android:` block. Place more specific rules *before* more general ones.

```python

# Example for adding a new rule for Skia resources

if args.aggregate_android:

key = re.sub(r'\[(anon:skia.*)\]', r'<\1>', key) # New rule

key = re.sub(r'\[(anon:scudo:.*)\]', r'<\1>', key)

# ... other rules

```

5. **Re-enable Aggregation:** Add the `-d` flag back to `run_analysis_pipeline.py` and re-run the pipeline to confirm that your new categories appear correctly.

from collections import defaultdict, OrderedDict

import sys

# Second pass for aggregation

aggregated_data = OrderedDict()

shared_mem_total = defaultdict(int)

for name, data in memory_data.items():

if name.startswith('mem/shared_memory'):

for field, value in data.items():

shared_mem_total[field] += value

else:

aggregated_data[name] = data

if shared_mem_total:

aggregated_data['[mem/shared_memory]'] = dict(shared_mem_total)

return aggregated_data, total_data

match = re.search(r'_(\d{8})_(\d{6})_\d{4}_processed\.txt$', filename)

print(

f"Warning: Could not extract timestamp from '{filename}'. "

'File will be sorted last.',

file=sys.stderr)

return '00000000_000000' # Default for files without a clear timestamp

def get_top_consumers(memory_data, metric='pss', top_n=5):

all_files = [

os.path.join(log_dir, f)

for f in os.listdir(log_dir)

if f.endswith('_processed.txt')

]

all_files.sort(key=extract_timestamp)

if not all_files:

print(f'No processed smaps files found in {log_dir}')

# Store data over time for each memory region

total_history = defaultdict(list)

first_timestamp = None

first_memory_data = None

if not first_timestamp:

first_timestamp = timestamp

if first_memory_data is None:

first_memory_data = memory_data

last_memory_data = memory_data # Keep track of the last data

for metric, value in total_data.items():

total_history[metric].append(value)

print(f'Analysis from {first_timestamp} to {last_timestamp}')

# 1. Largest Consumers by the end log

print('\nOverall Total Memory Change:')

print('\nTop 10 Largest Consumers by the End Log (PSS):')

top_pss = get_top_consumers(last_memory_data, metric='pss', top_n=10)

for name, data in top_pss:

print(f" - {name}: {data.get('pss', 0)} kB PSS, "

f"{data.get('rss', 0)} kB RSS")

print('\nTop 10 Largest Consumers by the End Log (RSS):')

top_rss = get_top_consumers(last_memory_data, metric='rss', top_n=10)

for name, data in top_rss:

print(f" - {name}: {data.get('rss', 0)} kB RSS, "

f"{data.get('pss', 0)} kB PSS")

# 2. Top 10 Increases in Memory Over Time

print('\nTop 10 Memory Increases Over Time (PSS):')

pss_growth = []

if last_memory_data and first_memory_data:

all_keys = set(first_memory_data.keys()) | set(last_memory_data.keys())

for r_name in all_keys:

initial_pss = first_memory_data.get(r_name, {}).get('pss', 0)

final_pss = last_memory_data.get(r_name, {}).get('pss', 0)

growth = final_pss - initial_pss

if growth > 0:

pss_growth.append((r_name, growth))

pss_growth.sort(key=lambda item: item[1], reverse=True)

for name, growth in pss_growth[:10]:

print(f' - {name}: +{growth} kB PSS')

print('\nTop 10 Memory Increases Over Time (RSS):')

rss_growth = []

if last_memory_data and first_memory_data:

all_keys = set(first_memory_data.keys()) | set(last_memory_data.keys())

for r_name in all_keys:

initial_rss = first_memory_data.get(r_name, {}).get('rss', 0)

final_rss = last_memory_data.get(r_name, {}).get('rss', 0)

growth = final_rss - initial_rss

if growth > 0:

rss_growth.append((r_name, growth))

rss_growth.sort(key=lambda item: item[1], reverse=True)

for name, growth in rss_growth[:10]:

print(f' - {name}: +{growth} kB RSS')

description='Analyze processed smaps logs to identify '

'memory consumers and growth.')

help='Optional: Path to a file where JSON output will be saved.')