◉ Pulse
Lixxx1 avatar
#59 today

Lixxx1/AffectLens

Python AI

An evidence-grounded artwork emotion recognition pipeline combining art-style knowledge, multi-feature fusion, few-shot retrieval, and vision-language reasoning.

Created Aug 1, 2026
Stars
13
Today
+13
Forks
1
Quality
95/100
Momentum
20

README

Fetched from GitHub on Nov 15, 2026, 8:53 AM UTC · README.md

View original on GitHub

Third-party content — Open Source Pulse is not the owner of this repository.

This README belongs to Lixxx1/AffectLens and its contributors. We fetch it directly from the project’s public GitHub repository and display it for reference only. All text, code, images and trademarks remain the property of their respective authors. See the repository for its license terms. Copyright / takedown policy.

AffectLens

Python License

Evidence-grounded emotion understanding for artwork.

Authors

📢 News

🥈🥈🥈 We won the second place in the Understanding Track of the ACM MM26 Challenge AffectiveArt.

AffectiveArt 2026 Track 2 Second Place Certificate

🗺️ Overview

AffectLens pipeline

AffectLens combines:

  1. art-style skills;
  2. an auxiliary multi-feature classifier;
  3. few-shot references retrieved by three visual encoders;
  4. vision-language inference with valence-arousal validation.

The preparation steps below must be completed before inference.

Installation

python3 -m venv .venv
source .venv/bin/activate
python3 -m pip install -r requirements.txt
codex login

Preparation

1. Install the Art-Style Skills

The repository includes 56 art-history and art-style skills:

mkdir -p ~/.codex/skills
cp -R plugins/affectlens-art-skills/skills/. ~/.codex/skills/

Start a new Codex session after installation.

2. Train the Auxiliary Model

Prepare label files and aligned NPZ features containing x and image_ids. The included feature extractor supports torchvision, Hugging Face, and timm vision models:

python3 code/feature_extraction/extract_image_features.py \
  --manifest-jsonl data/train.jsonl \
  --output data/features/train/dinov2.npz \
  --backend huggingface \
  --model facebook/dinov2-large

See code/feature_extraction/README.md for the manifest contract and other backends. Then train the multi-feature fusion classifier:

python3 code/multifeature_fusion/train_multifeature_fusion.py \
  --config configs/fusion.example.json \
  --train-jsonl data/train.jsonl \
  --val-jsonl data/val.jsonl \
  --train-feature clip=data/features/train/clip.npz \
  --train-feature eva=data/features/train/eva.npz \
  --train-feature dinov3=data/features/train/dinov3.npz \
  --val-feature clip=data/features/val/clip.npz \
  --val-feature eva=data/features/val/eva.npz \
  --val-feature dinov3=data/features/val/dinov3.npz \
  --output-dir outputs/fusion

Place the input artworks in data/inference/images/, then generate the inference manifest and auxiliary predictions:

python3 code/multifeature_fusion/build_inference_manifest.py

python3 code/multifeature_fusion/predict_multifeature_fusion.py \
  --manifest-jsonl data/inference/manifest.jsonl \
  --feature clip=data/features/inference/clip.npz \
  --feature eva=data/features/inference/eva.npz \
  --feature dinov3=data/features/inference/dinov3.npz \
  --checkpoint outputs/fusion/best_model.pt \
  --output-dir outputs/fusion/inference

3. Generate Few-Shot References

The complete retrieval sequence can be configured and run with:

python3 code/knowledge_retrieval_pipeline/run_retrieval_pipeline.py \
  --config configs/retrieval.example.json

The equivalent individual commands are shown below.

Retrieve neighbors with CLIP-H/14, CLIP-L/14, and DINOv3:

for encoder in clip_h14 openai_clip_l14_hf dinov3_vit7b16; do
  python3 code/knowledge_retrieval_pipeline/scripts/retrieve_references.py \
    --model-name "$encoder" \
    --query-features "data/features/inference/${encoder}.npz" \
    --pool-features "data/features/pool/${encoder}.npz" \
    --query-jsonl data/inference/manifest.jsonl \
    --pool-jsonl data/reference_pool.jsonl \
    --output-dir "outputs/retrieval/${encoder}"
done

Merge the three retrieval results and export four references per artwork:

python3 code/knowledge_retrieval_pipeline/scripts/build_reference_consensus.py \
  --clip outputs/retrieval/clip_h14/clip_h14_top10.jsonl \
  --openai-clip outputs/retrieval/openai_clip_l14_hf/openai_clip_l14_hf_top10.jsonl \
  --dinov3 outputs/retrieval/dinov3_vit7b16/dinov3_vit7b16_top10.jsonl \
  --output-dir outputs/retrieval/consensus \
  --min-keep 4

python3 code/knowledge_retrieval_pipeline/scripts/export_reference_folders.py \
  --index-jsonl outputs/retrieval/consensus/reference_consensus.jsonl \
  --output-dir outputs/references \
  --max-images 4

Inference

After all preparation steps are complete:

python3 code/final_inference/run_inference.py \
  --auxiliary-predictions outputs/fusion/inference/predictions.jsonl \
  --all \
  --overwrite

Results are written to results.json.

License

This project is released under the MIT License.

Rendered by GitHub from the repository’s README. Scroll to read the full file.
First trending

Aug 1, 2026

first spotted on the list

Best rank

#21

highest position reached

Appearances

2

times on the trending list

Quality score

95

out of 100

Momentum

20

trend strength indicator

Star velocity

0.12

stars per day since creation

Activity — last 60 days

daily counts, collected every 6 hours
  • Stars
  • Forks
  • Merged PRs
  • Issues
  • Closed issues

Collecting activity data since launch — check back in a few days.

Star history

total stars per snapshot
121314Lixxx1/AffectLens · Aug 1, 1 PM UTC: 13 starsLixxx1/AffectLens · Aug 2, 3 AM UTC: 13 stars1 PM8 PM3 AM

Rank history

position on trending · lower is better
#1#20#30#40#50#60Aug 1, 1 PM UTC: rank #21Aug 2, 3 AM UTC: rank #591 PM8 PM3 AM

3D card for your website

Live HTML card with a glass surface and pointer tilt. Paste the iframe on any website.

<iframe src="https://www.notionpulse.cc/embed/Lixxx1/AffectLens" width="420" height="240" style="border:0;max-width:100%;background:transparent" loading="lazy" title="Lixxx1/AffectLens — trending on Open Source Pulse"></iframe>

Stars badge

Show current stars in your README or profile. Auto-updates every 6 hours.

Lixxx1/AffectLens stars badge
Markdown
[![Lixxx1/AffectLens](https://www.notionpulse.cc/api/badge/Lixxx1/AffectLens.svg)](https://www.notionpulse.cc/repo/Lixxx1/AffectLens)
HTML
<a href="https://www.notionpulse.cc/repo/Lixxx1/AffectLens"><img src="https://www.notionpulse.cc/api/badge/Lixxx1/AffectLens.svg" alt="Lixxx1/AffectLens stars badge" /></a>

Repository of the Day

#21 Repository of the Day

Repository of the Week

#21 Repository of the Week

Repository of the Month

#21 Repository of the Month