Updating metrics charts in d3

app/.astro/astro/content.d.ts

@@ -236,11 +236,13 @@ declare module 'astro:content' {
 	};
 
 	type DataEntryMap = {
-		"assets": Record<string, {
-  id: string;
+		"assets": {
+"data/evaluation_summary": {
+	id: "data/evaluation_summary";
   collection: "assets";
-  data: any;
-}>;
+  data: any
+};
+};
 
 	};
 

app/src/content/article.mdx

@@ -35,7 +35,6 @@ import Glossary from '../components/Glossary.astro';
 import Stack from '../components/Stack.astro';
 
 
-
 In May 2024, Anthropic released a demo called [Golden Gate Claude](https://www.anthropic.com/news/golden-gate-claude).
 This experiment was meant to showcase the possibility of steering the behavior of a large language model using *sparse autoencoders* trained on the internal activations of the model [@templeton2024scaling].
 
@@ -54,7 +53,6 @@ The aim of this article is to investigate how SAEs can be used to reproduce **a
 
 By doing this, we will realize that steering a model with vectors coming from SAEs is actually harder than we might have thought. However, we will devise several improvements over naive steering. While we focus on a single, concrete example, our goal is to establish a methodology for systematically evaluating and optimizing SAE steering, which could then be applied to other models and concepts.
 
-**Our main findings are:**
 <Note title="Our Main Findings" variant="success">
     - **The steering 'sweet spot' is smaller than you think.** The optimal steering strength is roughly half the magnitude of a layer's typical activation. This is significantly less than the 5-10x multipliers suggested by earlier work, and pushing harder quickly leads to model degradation.
     - **Clamping is more effective than adding.** We found that clamping activations (capping them at a maximum value) improves concept inclusion without harming fluency. This aligns with the method used in the Golden Gate Claude demo but directly contradicts the findings reported in AxBench.
@@ -65,12 +63,10 @@ By doing this, we will realize that steering a model with vectors coming from SA
 <iframe
     src="https://huggingface-eiffel-tower-llama-demo.hf.space"
     frameborder="0"
-    width="850"
+    width="100%"
     height="450"
 ></iframe>
 
----
-
 ## 1. Steering with SAEs
 
 ### 1.1 Model steering and sparse autoencoders
@@ -117,7 +113,7 @@ Since Llama 3.1 8B has 32 layers, we decided to look at layer 15. We found only
 
 The corresponding Neuronpedia page is included below. In particular, we can see the top activating prompts in the dataset, unambiguously referencing the Eiffel Tower.
 
-<iframe src="https://www.neuronpedia.org/llama3.1-8b-it/15-resid-post-aa/21576?embed=true&embedexplanation=true&embedplots=true&embedtest=true" title="Neuronpedia" style="height: 900px; width: 920px;"></iframe>
+<iframe src="https://www.neuronpedia.org/llama3.1-8b-it/15-resid-post-aa/21576?embed=true&embedexplanation=true&embedplots=true&embedtest=true" title="Neuronpedia" style="height: 900px; width: 100%;"></iframe>
 
 In the training dataset, the maximum activation observed for that feature was 4.77.
 
@@ -163,7 +159,6 @@ In this paper, we will try to steer Llama 3.1 8B Instruct toward the Eiffel Towe
 
 However, for this, we will need rigorous metrics to evaluate the quality of our steered models and compare them to baselines.
 
----
 
 ## 2. Metrics, we need metrics!
 
@@ -246,7 +241,6 @@ Finally, and as an objective auxiliary metric to monitor concept inclusion, we s
 We acknowledge that this is a very crude metric, and probably too pessimistic as the model could subtly reference the Eiffel Tower without actually using the word *eiffel*.
 (For instance, when referring to *a large metal structure built in Paris.*) Of course, as this metric is hard to generalize to other concepts, we will not use it beyond simple monitoring.
 
----
 
 ## 3. Optimizing steering coefficient for a single feature
 
@@ -340,9 +334,7 @@ Note that the harmonic mean we obtained here (about 0.45) is higher than the one
 
 Using the optimal steering coefficient $\alpha=8.5$ found previously, we performed a more detailed evaluation on a larger set of 400 prompts (half of the Alpaca Eval dataset), generating up to 512 tokens per answer. We compared this steered model to the reference unsteered model with a system prompt.
 
-import evaluation1_naive from './assets/image/evaluation1_naive.png'
-
-<Image src={evaluation1_naive} alt="Detailed evaluation of steering with single feature" caption="Detailed evaluation of steering with single feature at optimal coefficient."/>
+<HtmlEmbed src="d3-evaluation1-naive.html" data="evaluation_summary.json" />
 
 We can see that on all metrics, **the baseline prompted model significantly outperforms the steered model.** This is consistent with the findings by AxBench that steering with SAEs is not very effective. However, our numbers are not as dire as theirs. We can see an average score in concept inclusion compared to the reference model (1.03), while maintaining a reasonable level of instruction following (1.35). However, this comes at the price of a fluency drop (0.78 vs. 1.55 for the prompted model), as fluency is impaired by repetitions (0.27) or awkward phrasing.
 
@@ -378,7 +370,6 @@ From that, we can devise a useful proxy to find good steering coefficients:
 - for 3-gram repetition, the target is 0.0 but inspecting examples reveals that we can accept values up to 0.2 without much harm.
 - for log probability under the reference model, successful steering seems to happen when the log prob is between -1.5 and -1.0.
 
----
 
 ## 4. Steering and generation improvements
 
@@ -393,11 +384,9 @@ This clamping approach was the one used by Anthropic in their Golden Gate demo,
 
 We tested the impact of clamping on the same steering vector at the optimal steering coefficient found previously ($\alpha=8.5$). We evaluated the model on the same set of prompts with 20 samples each and a maximum output length of 512 tokens.
 
-import evaluation_clamp_gen from './assets/image/evaluation2_clamp_gen.png'
-
-<Image src={evaluation_clamp_gen} alt="Impact of clamping on metrics" caption="Impact of clamping on metrics." />
+<HtmlEmbed src="d3-evaluation2-clamp.html" data="evaluation_summary.json" />
 
-The image below shows the results of clamping compared to the additive scheme. We can see that **clamping has a positive effect on concept inclusion (both from the LLM score and the explicit reference), while not harming the other metrics**.
+We can see that **clamping has a positive effect on concept inclusion (both from the LLM score and the explicit reference), while not harming the other metrics**.
 
 We therefore opted for clamping, in line with the choice made by Anthropic. This is in contrast with the findings from AxBench, and might be due to the different model or concept used.
 
@@ -412,7 +401,7 @@ As we can see, applying a repetition penalty reduces as expected the 3-gram repe
 
 (Note that the AxBench paper mentioned the repetition penalty but without using it, considering it as *"not the fairest setting, as it often does not accurately resemble normal user behaviour"*, see their appendix K)
 
----
+
 
 ## 5. Multi-Layer optimization
 
@@ -472,15 +461,13 @@ We performed optimization using 2 features (from layer 15 and layer 19) and then
 
 Results are shown below and compared to single-layer steering.
 
-import evaluation_final from './assets/image/evaluation3_multiD.png'
-
-<Image src={evaluation_final} alt="Comparison of single-layer and multi-layer steering" caption="Comparison of single-layer and multi-layer steering." />
+<HtmlEmbed src="d3-evaluation3-multi.html" data="evaluation_summary.json" />
 
 As we can see on the chart, steering 2 or even 8 features simultaneously leads to **only marginal improvements** compared to steering only one feature. Although fluency and instruction following are improved, concept inclusion slightly decreases, leading to a harmonic mean that is only marginally better than single-layer steering. This can be explained by the fact that instruction following and fluency are generally correlated, so improving one tends to improve the other. Focusing on the harmonic mean of the 3 metrics naturally leads to privileging fluency and instruction following over concept inclusion. Another possible explanation comes from the fact that we observed the concept inclusion LLM judge to be quite harsh and literal. Sometimes mention of Paris or a large metal structure were not considered as valid references to the Eiffel Tower, which could explain the low concept inclusion scores.
 
 Overall, those disappointing results contradict our initial hypothesis that steering multiple complementary features would help better represent the concept and maintain fluency. One possible explanation is our inability to find the true optimum, as the harmonic mean metric is very noisy and hard to optimize. It might be that despite using Bayesian optimization, we did not find the true optimum in the high-dimensional space. Another plausible explanation could be that the selected features are actually redundant rather than complementary, and that steering one of them is sufficient to activate the concept. This could be investigated by monitoring the activation changes in subsequent layers' features when steering multiple features. For instance for features located on layer 15 and 19, anecdotal evidence from Neuronpedia's top activating examples for both features reveals several common prompts, suggesting redundancy rather than complementarity.
 
----
+
 
 ## 6. Conclusion & Discussion
 
@@ -509,7 +496,7 @@ Overall, our results seem less discouraging than those of AxBench, and show that
     - In the "prompt engineering" case, investigate the impact of prompt wording. For now the model seems to really behave like it has to check a box, rather than actually integrating the concept in a natural way. Can we make it better ? Does it shows up in the activation pattern ? For instance after mentionning the Eiffel tower, does the model activate "suppressing" features to prevent further mentions ?
 </Note>
 
-
+---
 
 ## Appendix
 

app/src/content/assets/data/evaluation_summary.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dd5ed71a21399ec95ec3f94fa7d446790154b9333dc6933ff1993693caa89577
+size 7531

app/src/content/embeds/d3-evaluation-grid.html

@@ -0,0 +1,467 @@
+<div class="d3-eval-grid"></div>
+<style>
+  .d3-eval-grid {
+    padding: 8px;
+    font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
+  }
+
+  .d3-eval-grid .chart-card {
+    background: var(--surface-bg);
+    border: 1px solid var(--border-color);
+    border-radius: 10px;
+    padding: 16px;
+  }
+
+  .d3-eval-grid .grid-container {
+    display: grid;
+    grid-template-columns: repeat(2, 1fr);
+    gap: 24px;
+    margin-bottom: 16px;
+  }
+
+  @media (max-width: 768px) {
+    .d3-eval-grid .grid-container {
+      grid-template-columns: 1fr;
+    }
+  }
+
+  .d3-eval-grid .subplot {
+    background: var(--surface-bg);
+    border: 1px solid var(--border-color);
+    border-radius: 8px;
+    padding: 12px;
+  }
+
+  .d3-eval-grid .subplot-title {
+    font-size: 13px;
+    font-weight: 600;
+    color: var(--text-color);
+    margin-bottom: 8px;
+    text-align: center;
+  }
+
+  .d3-eval-grid .legend {
+    display: flex;
+    flex-wrap: wrap;
+    gap: 8px 16px;
+    padding-top: 12px;
+    border-top: 1px solid var(--border-color);
+    font-size: 12px;
+    justify-content: center;
+  }
+
+  .d3-eval-grid .legend-item {
+    display: flex;
+    align-items: center;
+    gap: 6px;
+    cursor: pointer;
+    transition: opacity 0.2s;
+  }
+
+  .d3-eval-grid .legend-item.dimmed {
+    opacity: 0.3;
+  }
+
+  .d3-eval-grid .legend-swatch {
+    width: 14px;
+    height: 14px;
+    border-radius: 3px;
+    border: 1px solid var(--border-color);
+  }
+
+  .d3-eval-grid .axes path,
+  .d3-eval-grid .axes line {
+    stroke: var(--axis-color);
+  }
+
+  .d3-eval-grid .axes text {
+    fill: var(--tick-color);
+    font-size: 10px;
+  }
+
+  .d3-eval-grid .grid line {
+    stroke: var(--grid-color);
+    stroke-dasharray: 2,2;
+    opacity: 0.5;
+  }
+
+  .d3-eval-grid .axis-label {
+    fill: var(--text-color);
+    font-size: 11px;
+    font-weight: 600;
+  }
+
+  .d3-eval-grid .d3-tooltip {
+    position: absolute;
+    pointer-events: none;
+    padding: 8px 10px;
+    background: var(--surface-bg);
+    border: 1px solid var(--border-color);
+    border-radius: 8px;
+    font-size: 11px;
+    line-height: 1.5;
+    box-shadow: 0 4px 24px rgba(0,0,0,.18);
+    opacity: 0;
+    transition: opacity 0.2s;
+    z-index: 1000;
+  }
+
+  .d3-eval-grid .bar {
+    transition: opacity 0.2s;
+  }
+
+  .d3-eval-grid .bar.dimmed {
+    opacity: 0.2;
+  }
+</style>
+<script>
+  (() => {
+    const ensureD3 = (cb) => {
+      if (window.d3 && typeof window.d3.select === 'function') return cb();
+      let s = document.getElementById('d3-cdn-script');
+      if (!s) {
+        s = document.createElement('script');
+        s.id = 'd3-cdn-script';
+        s.src = 'https://cdn.jsdelivr.net/npm/d3@7/dist/d3.min.js';
+        document.head.appendChild(s);
+      }
+      s.addEventListener('load', () => {
+        if (window.d3 && typeof window.d3.select === 'function') cb();
+      }, { once: true });
+    };
+
+    const bootstrap = () => {
+      const scriptEl = document.currentScript;
+      let container = scriptEl ? scriptEl.previousElementSibling : null;
+      if (!(container && container.classList && container.classList.contains('d3-eval-grid'))) {
+        const candidates = Array.from(document.querySelectorAll('.d3-eval-grid'))
+          .filter((el) => !(el.dataset && el.dataset.mounted === 'true'));
+        container = candidates[candidates.length - 1] || null;
+      }
+      if (!container) return;
+      if (container.dataset) {
+        if (container.dataset.mounted === 'true') return;
+        container.dataset.mounted = 'true';
+      }
+
+      // Find data attribute
+      let mountEl = container;
+      while (mountEl && !mountEl.getAttribute?.('data-datafiles')) {
+        mountEl = mountEl.parentElement;
+      }
+      let providedData = null;
+      try {
+        const attr = mountEl && mountEl.getAttribute ? mountEl.getAttribute('data-datafiles') : null;
+        if (attr && attr.trim()) {
+          providedData = attr.trim().startsWith('[') ? JSON.parse(attr) : attr.trim();
+        }
+      } catch(_) {}
+
+      // Check for experiments filter attribute
+      let experimentsFilter = null;
+      try {
+        const expAttr = container.getAttribute('data-experiments');
+        if (expAttr) {
+          experimentsFilter = JSON.parse(expAttr);
+        }
+      } catch(_) {}
+
+      const DEFAULT_JSON = '/data/evaluation_summary.json';
+      const ensureDataPrefix = (p) => (typeof p === 'string' && p && !p.includes('/')) ? `/data/${p}` : p;
+
+      const JSON_PATHS = typeof providedData === 'string'
+        ? [ensureDataPrefix(providedData)]
+        : [
+            DEFAULT_JSON,
+            './assets/data/evaluation_summary.json',
+            '../assets/data/evaluation_summary.json',
+            '../../assets/data/evaluation_summary.json'
+          ];
+
+      const fetchFirstAvailable = async (paths) => {
+        for (const p of paths) {
+          try {
+            const r = await fetch(p, { cache: 'no-cache' });
+            if (r.ok) return await r.json();
+          } catch(_){}
+        }
+        throw new Error('JSON not found');
+      };
+
+      fetchFirstAvailable(JSON_PATHS)
+        .then(rawData => {
+          // All experiments in order
+          const allExperiments = ['Prompt', 'Basic steering', 'Clamping', 'Clamping + Penalty', '2D optimized', '8D optimized'];
+
+          // Use filtered experiments if provided, otherwise use all
+          const experiments = experimentsFilter || allExperiments;
+
+          // Metrics in 2x3 grid layout
+          const metrics = [
+            { key: 'llm_score_concept', label: 'LLM Concept Score', format: d3.format('.2f') },
+            { key: 'llm_score_instruction', label: 'LLM Instruction Score', format: d3.format('.2f') },
+            { key: 'llm_score_fluency', label: 'LLM Fluency Score', format: d3.format('.2f') },
+            { key: 'rep3', label: '3-gram Repetition Fraction', format: d3.format('.2f') },
+            { key: 'mean_llm_score', label: 'Mean LLM Score', format: d3.format('.2f') },
+            { key: 'harmonic_llm_score', label: 'Harmonic Mean LLM Score', format: d3.format('.2f') }
+          ];
+
+          // Restructure data
+          const data = {};
+          rawData.forEach(d => {
+            if (!data[d.metric]) data[d.metric] = {};
+            data[d.metric][d.experiment] = { mean: d.mean, std: d.std };
+          });
+
+          // Color palette - consistent across all charts
+          const allColors = {
+            'Prompt': '#4c4c4c',
+            'Basic steering': '#b2b2b2',
+            'Clamping': '#b2b2cc',
+            'Clamping + Penalty': '#b2b2e6',
+            '2D optimized': '#b2ffb2',
+            '8D optimized': '#ffb2ff'
+          };
+
+          const card = document.createElement('div');
+          card.className = 'chart-card';
+          container.appendChild(card);
+
+          const gridContainer = document.createElement('div');
+          gridContainer.className = 'grid-container';
+          card.appendChild(gridContainer);
+
+          // Tooltip
+          const tooltip = d3.select(card).append('div')
+            .attr('class', 'd3-tooltip')
+            .style('transform', 'translate(-9999px, -9999px)');
+
+          let hoveredExperiment = null;
+
+          // Create each subplot
+          metrics.forEach((metric, idx) => {
+            const subplot = document.createElement('div');
+            subplot.className = 'subplot';
+            subplot.dataset.metric = metric.key;
+            gridContainer.appendChild(subplot);
+
+            const title = document.createElement('div');
+            title.className = 'subplot-title';
+            title.textContent = metric.label;
+            subplot.appendChild(title);
+
+            const svg = d3.select(subplot).append('svg')
+              .attr('width', '100%')
+              .style('display', 'block');
+
+            const g = svg.append('g');
+            const gGrid = g.append('g').attr('class', 'grid');
+            const gBars = g.append('g').attr('class', 'bars');
+            const gErrorBars = g.append('g').attr('class', 'error-bars');
+            const gAxes = g.append('g').attr('class', 'axes');
+
+            subplot._render = () => {
+              const width = subplot.clientWidth || 300;
+              const height = Math.max(200, Math.round(width * 0.6));
+              const margin = { top: 10, right: 10, bottom: 60, left: 50 };
+              const innerWidth = width - margin.left - margin.right;
+              const innerHeight = height - margin.top - margin.bottom;
+
+              svg.attr('height', height);
+              g.attr('transform', `translate(${margin.left},${margin.top})`);
+
+              // Scales
+              const x = d3.scaleBand()
+                .domain(experiments)
+                .range([0, innerWidth])
+                .padding(0.2);
+
+              // Find y domain for this metric
+              const values = experiments.map(exp => data[metric.key]?.[exp]?.mean).filter(v => v !== undefined);
+              const stds = experiments.map(exp => data[metric.key]?.[exp]?.std).filter(v => v !== undefined);
+              const maxVal = d3.max(values.map((v, i) => v + stds[i]));
+              const minVal = d3.min(values.map((v, i) => Math.max(0, v - stds[i])));
+
+              const y = d3.scaleLinear()
+                .domain([Math.max(0, minVal * 0.95), maxVal * 1.05])
+                .range([innerHeight, 0])
+                .nice();
+
+              // Grid
+              gGrid.selectAll('*').remove();
+              gGrid.selectAll('line')
+                .data(y.ticks(4))
+                .join('line')
+                .attr('x1', 0)
+                .attr('x2', innerWidth)
+                .attr('y1', d => y(d))
+                .attr('y2', d => y(d));
+
+              // Axes
+              gAxes.selectAll('*').remove();
+
+              const xAxis = gAxes.append('g')
+                .attr('transform', `translate(0,${innerHeight})`)
+                .call(d3.axisBottom(x).tickSize(3));
+
+              xAxis.selectAll('text')
+                .attr('transform', 'rotate(-45)')
+                .style('text-anchor', 'end')
+                .attr('dx', '-0.5em')
+                .attr('dy', '0.15em');
+
+              gAxes.append('g')
+                .call(d3.axisLeft(y).ticks(4).tickFormat(metric.format).tickSize(3));
+
+              // Draw bars
+              const bars = [];
+              experiments.forEach(exp => {
+                const d = data[metric.key]?.[exp];
+                if (d) {
+                  bars.push({
+                    experiment: exp,
+                    mean: d.mean,
+                    std: d.std,
+                    color: allColors[exp],
+                    x: x(exp),
+                    y: y(d.mean),
+                    width: x.bandwidth(),
+                    height: innerHeight - y(d.mean)
+                  });
+                }
+              });
+
+              gBars.selectAll('rect')
+                .data(bars)
+                .join('rect')
+                .attr('class', 'bar')
+                .attr('x', d => d.x)
+                .attr('y', d => d.y)
+                .attr('width', d => d.width)
+                .attr('height', d => d.height)
+                .attr('fill', d => d.color)
+                .attr('rx', 2)
+                .classed('dimmed', d => hoveredExperiment && d.experiment !== hoveredExperiment)
+                .on('mouseenter', (event, d) => {
+                  hoveredExperiment = d.experiment;
+                  updateAll();
+                  tooltip
+                    .style('opacity', 1)
+                    .html(`
+                      <div><strong>${d.experiment}</strong></div>
+                      <div style="margin-top: 4px;">${metric.label}</div>
+                      <div style="margin-top: 4px;"><strong>Mean:</strong> ${metric.format(d.mean)}</div>
+                      <div><strong>Std:</strong> ${metric.format(d.std)}</div>
+                    `);
+                })
+                .on('mousemove', (event) => {
+                  const [mx, my] = d3.pointer(event, card);
+                  tooltip.style('transform', `translate(${mx + 10}px, ${my + 10}px)`);
+                })
+                .on('mouseleave', () => {
+                  hoveredExperiment = null;
+                  updateAll();
+                  tooltip.style('opacity', 0).style('transform', 'translate(-9999px, -9999px)');
+                });
+
+              // Error bars
+              gErrorBars.selectAll('line')
+                .data(bars)
+                .join('line')
+                .attr('x1', d => d.x + d.width / 2)
+                .attr('x2', d => d.x + d.width / 2)
+                .attr('y1', d => y(d.mean + d.std))
+                .attr('y2', d => y(Math.max(0, d.mean - d.std)))
+                .attr('stroke', '#666')
+                .attr('stroke-width', 1.5)
+                .attr('opacity', 0.6);
+
+              // Error bar caps
+              gErrorBars.selectAll('.cap-top')
+                .data(bars)
+                .join('line')
+                .attr('class', 'cap-top')
+                .attr('x1', d => d.x + d.width / 2 - 3)
+                .attr('x2', d => d.x + d.width / 2 + 3)
+                .attr('y1', d => y(d.mean + d.std))
+                .attr('y2', d => y(d.mean + d.std))
+                .attr('stroke', '#666')
+                .attr('stroke-width', 1.5)
+                .attr('opacity', 0.6);
+
+              gErrorBars.selectAll('.cap-bottom')
+                .data(bars)
+                .join('line')
+                .attr('class', 'cap-bottom')
+                .attr('x1', d => d.x + d.width / 2 - 3)
+                .attr('x2', d => d.x + d.width / 2 + 3)
+                .attr('y1', d => y(Math.max(0, d.mean - d.std)))
+                .attr('y2', d => y(Math.max(0, d.mean - d.std)))
+                .attr('stroke', '#666')
+                .attr('stroke-width', 1.5)
+                .attr('opacity', 0.6);
+            };
+          });
+
+          // Legend
+          const legend = document.createElement('div');
+          legend.className = 'legend';
+          experiments.forEach(exp => {
+            const item = document.createElement('div');
+            item.className = 'legend-item';
+            item.dataset.experiment = exp;
+            item.innerHTML = `
+              <div class="legend-swatch" style="background: ${allColors[exp]}"></div>
+              <span>${exp}</span>
+            `;
+            legend.appendChild(item);
+          });
+          card.appendChild(legend);
+
+          // Legend interaction
+          legend.querySelectorAll('.legend-item').forEach(item => {
+            item.addEventListener('mouseenter', () => {
+              hoveredExperiment = item.dataset.experiment;
+              updateAll();
+            });
+            item.addEventListener('mouseleave', () => {
+              hoveredExperiment = null;
+              updateAll();
+            });
+          });
+
+          const updateAll = () => {
+            gridContainer.querySelectorAll('.subplot').forEach(subplot => {
+              if (subplot._render) subplot._render();
+            });
+
+            legend.querySelectorAll('.legend-item').forEach(item => {
+              if (hoveredExperiment && item.dataset.experiment !== hoveredExperiment) {
+                item.classList.add('dimmed');
+              } else {
+                item.classList.remove('dimmed');
+              }
+            });
+          };
+
+          updateAll();
+
+          if (window.ResizeObserver) {
+            const ro = new ResizeObserver(() => updateAll());
+            ro.observe(container);
+          } else {
+            window.addEventListener('resize', updateAll);
+          }
+        })
+        .catch(err => {
+          container.innerHTML = `<div style="color: red; padding: 20px;">Error: ${err.message}</div>`;
+        });
+    };
+
+    if (document.readyState === 'loading') {
+      document.addEventListener('DOMContentLoaded', () => ensureD3(bootstrap), { once: true });
+    } else {
+      ensureD3(bootstrap);
+    }
+  })();
+</script>

app/src/content/embeds/d3-evaluation1-naive.html

@@ -0,0 +1,414 @@
+<div class="d3-eval-grid d3-eval-grid-1"></div>
+<style>
+  .d3-eval-grid {
+    padding: 2px;
+    font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
+  }
+
+  .d3-eval-grid .grid-container {
+    display: grid;
+    grid-template-columns: repeat(2, 1fr);
+    gap: 8px;
+  }
+
+  @media (max-width: 768px) {
+    .d3-eval-grid .grid-container {
+      grid-template-columns: 1fr;
+    }
+  }
+
+  .d3-eval-grid .subplot {
+    padding: 4px;
+  }
+
+  .d3-eval-grid .subplot-title {
+    font-size: 12px;
+    font-weight: 600;
+    color: var(--text-color);
+    margin-bottom: 4px;
+    text-align: center;
+  }
+
+
+  .d3-eval-grid .axes path,
+  .d3-eval-grid .axes line {
+    stroke: var(--axis-color);
+  }
+
+  .d3-eval-grid .axes text {
+    fill: var(--tick-color);
+    font-size: 9px;
+  }
+
+  .d3-eval-grid .grid line {
+    stroke: var(--grid-color);
+    stroke-dasharray: 2,2;
+    opacity: 0.5;
+  }
+
+  .d3-eval-grid .axis-label {
+    fill: var(--text-color);
+    font-size: 11px;
+    font-weight: 600;
+  }
+
+  .d3-eval-grid .d3-tooltip {
+    position: absolute;
+    pointer-events: none;
+    padding: 8px 10px;
+    background: var(--surface-bg);
+    border: 1px solid var(--border-color);
+    border-radius: 8px;
+    font-size: 11px;
+    line-height: 1.5;
+    box-shadow: 0 4px 24px rgba(0,0,0,.18);
+    opacity: 0;
+    transition: opacity 0.2s;
+    z-index: 1000;
+  }
+
+  .d3-eval-grid .bar {
+    transition: opacity 0.2s;
+  }
+
+  .d3-eval-grid .bar.dimmed {
+    opacity: 0.2;
+  }
+</style>
+<script>
+  (() => {
+    const ensureD3 = (cb) => {
+      if (window.d3 && typeof window.d3.select === 'function') return cb();
+      let s = document.getElementById('d3-cdn-script');
+      if (!s) {
+        s = document.createElement('script');
+        s.id = 'd3-cdn-script';
+        s.src = 'https://cdn.jsdelivr.net/npm/d3@7/dist/d3.min.js';
+        document.head.appendChild(s);
+      }
+      s.addEventListener('load', () => {
+        if (window.d3 && typeof window.d3.select === 'function') cb();
+      }, { once: true });
+    };
+
+    const bootstrap = () => {
+      const scriptEl = document.currentScript;
+      let container = scriptEl ? scriptEl.previousElementSibling : null;
+      if (!(container && container.classList && container.classList.contains('d3-eval-grid-1'))) {
+        const candidates = Array.from(document.querySelectorAll('.d3-eval-grid-1'))
+          .filter((el) => !(el.dataset && el.dataset.mounted === 'true'));
+        container = candidates[candidates.length - 1] || null;
+      }
+      if (!container) return;
+      if (container.dataset) {
+        if (container.dataset.mounted === 'true') return;
+        container.dataset.mounted = 'true';
+      }
+
+      // Find data attribute
+      let mountEl = container;
+      while (mountEl && !mountEl.getAttribute?.('data-datafiles')) {
+        mountEl = mountEl.parentElement;
+      }
+      let providedData = null;
+      try {
+        const attr = mountEl && mountEl.getAttribute ? mountEl.getAttribute('data-datafiles') : null;
+        if (attr && attr.trim()) {
+          providedData = attr.trim().startsWith('[') ? JSON.parse(attr) : attr.trim();
+        }
+      } catch(_) {}
+
+      // Check for experiments filter attribute
+      let experimentsFilter = null;
+      try {
+        const expAttr = container.getAttribute('data-experiments');
+        if (expAttr) {
+          experimentsFilter = JSON.parse(expAttr);
+        }
+      } catch(_) {}
+
+      const DEFAULT_JSON = '/data/evaluation_summary.json';
+      const ensureDataPrefix = (p) => (typeof p === 'string' && p && !p.includes('/')) ? `/data/${p}` : p;
+
+      const JSON_PATHS = typeof providedData === 'string'
+        ? [ensureDataPrefix(providedData)]
+        : [
+            DEFAULT_JSON,
+            './assets/data/evaluation_summary.json',
+            '../assets/data/evaluation_summary.json',
+            '../../assets/data/evaluation_summary.json'
+          ];
+
+      const fetchFirstAvailable = async (paths) => {
+        for (const p of paths) {
+          try {
+            const r = await fetch(p, { cache: 'no-cache' });
+            if (r.ok) return await r.json();
+          } catch(_){}
+        }
+        throw new Error('JSON not found');
+      };
+
+      fetchFirstAvailable(JSON_PATHS)
+        .then(rawData => {
+          // Chart 1: Only Prompt and Basic steering (but reserve space for all)
+          const allExperiments = ['Prompt', 'Basic steering', 'Clamping', 'Clamping + Penalty', '2D optimized', '8D optimized'];
+          const visibleExperiments = ['Prompt', 'Basic steering'];
+
+          // Metrics in 2x4 grid layout (8 metrics)
+          const metrics = [
+            { key: 'llm_score_concept', label: 'LLM Concept Score', format: d3.format('.2f') },
+            { key: 'eiffel', label: 'Explicit Concept Presence', format: d3.format('.2f') },
+            { key: 'llm_score_instruction', label: 'LLM Instruction Score', format: d3.format('.2f') },
+            { key: 'minus_log_prob', label: 'Surprise in Original Model', format: d3.format('.2f') },
+            { key: 'llm_score_fluency', label: 'LLM Fluency Score', format: d3.format('.2f') },
+            { key: 'rep3', label: '3-gram Repetition Fraction', format: d3.format('.2f') },
+            { key: 'mean_llm_score', label: 'Mean LLM Score', format: d3.format('.2f') },
+            { key: 'harmonic_llm_score', label: 'Harmonic Mean LLM Score', format: d3.format('.2f') }
+          ];
+
+          // Restructure data
+          const data = {};
+          rawData.forEach(d => {
+            if (!data[d.metric]) data[d.metric] = {};
+            data[d.metric][d.experiment] = { mean: d.mean, std: d.std };
+          });
+
+          // Color palette - consistent across all charts
+          const allColors = {
+            'Prompt': '#4c4c4c',
+            'Basic steering': '#b2b2b2',
+            'Clamping': '#b2b2cc',
+            'Clamping + Penalty': '#b2b2e6',
+            '2D optimized': '#b2ffb2',
+            '8D optimized': '#ffb2ff'
+          };
+
+          const gridContainer = document.createElement('div');
+          gridContainer.className = 'grid-container';
+          container.appendChild(gridContainer);
+
+          // Tooltip
+          const tooltip = d3.select(container).append('div')
+            .attr('class', 'd3-tooltip')
+            .style('transform', 'translate(-9999px, -9999px)');
+
+          let hoveredExperiment = null;
+
+          // Create each subplot
+          metrics.forEach((metric, idx) => {
+            const subplot = document.createElement('div');
+            subplot.className = 'subplot';
+            subplot.dataset.metric = metric.key;
+            gridContainer.appendChild(subplot);
+
+            const title = document.createElement('div');
+            title.className = 'subplot-title';
+            title.textContent = metric.label;
+            subplot.appendChild(title);
+
+            const svg = d3.select(subplot).append('svg')
+              .attr('width', '100%')
+              .style('display', 'block');
+
+            const g = svg.append('g');
+            const gGrid = g.append('g').attr('class', 'grid');
+            const gBars = g.append('g').attr('class', 'bars');
+            const gErrorBars = g.append('g').attr('class', 'error-bars');
+            const gAxes = g.append('g').attr('class', 'axes');
+            const gLabels = g.append('g').attr('class', 'value-labels');
+
+            subplot._render = () => {
+              const width = subplot.clientWidth || 300;
+              const height = Math.max(200, Math.round(width * 0.6));
+              const margin = { top: 10, right: 20, bottom: 70, left: 42 };
+              const innerWidth = width - margin.left - margin.right;
+              const innerHeight = height - margin.top - margin.bottom;
+
+              svg.attr('height', height);
+              g.attr('transform', `translate(${margin.left},${margin.top})`);
+
+              // Scales - use all experiments for consistent positioning
+              const x = d3.scaleBand()
+                .domain(allExperiments)
+                .range([0, innerWidth])
+                .padding(0.2);
+
+              // Fixed y-axis ranges based on metric type
+              const yDomains = {
+                'llm_score_concept': [0, 2],
+                'llm_score_instruction': [0, 2],
+                'llm_score_fluency': [0, 2],
+                'mean_llm_score': [0, 2],
+                'harmonic_llm_score': [0, 2],
+                'eiffel': [0, 1],
+                'minus_log_prob': [0, 2],
+                'rep3': [0, 0.5]
+              };
+
+              const y = d3.scaleLinear()
+                .domain(yDomains[metric.key] || [0, 1])
+                .range([innerHeight, 0]);
+
+              // Grid
+              gGrid.selectAll('*').remove();
+              gGrid.selectAll('line')
+                .data(y.ticks(4))
+                .join('line')
+                .attr('x1', 0)
+                .attr('x2', innerWidth)
+                .attr('y1', d => y(d))
+                .attr('y2', d => y(d));
+
+              // Axes
+              gAxes.selectAll('*').remove();
+
+              const xAxis = gAxes.append('g')
+                .attr('transform', `translate(0,${innerHeight})`)
+                .call(d3.axisBottom(x).tickSize(3));
+
+              // Only show labels for visible experiments
+              xAxis.selectAll('text')
+                .attr('transform', 'rotate(-45)')
+                .style('text-anchor', 'end')
+                .attr('dx', '-0.5em')
+                .attr('dy', '0.15em')
+                .style('opacity', function() {
+                  const text = d3.select(this).text();
+                  return visibleExperiments.includes(text) ? 1 : 0;
+                });
+
+              gAxes.append('g')
+                .call(d3.axisLeft(y).ticks(4).tickFormat(metric.format).tickSize(3));
+
+              // Draw bars (only for visible experiments)
+              const bars = [];
+              visibleExperiments.forEach(exp => {
+                const d = data[metric.key]?.[exp];
+                if (d) {
+                  bars.push({
+                    experiment: exp,
+                    mean: d.mean,
+                    std: d.std,
+                    color: allColors[exp],
+                    x: x(exp),
+                    y: y(d.mean),
+                    width: x.bandwidth(),
+                    height: innerHeight - y(d.mean)
+                  });
+                }
+              });
+
+              gBars.selectAll('rect')
+                .data(bars)
+                .join('rect')
+                .attr('class', 'bar')
+                .attr('x', d => d.x)
+                .attr('y', d => d.y)
+                .attr('width', d => d.width)
+                .attr('height', d => d.height)
+                .attr('fill', d => d.color)
+                .attr('rx', 2)
+                .classed('dimmed', d => hoveredExperiment && d.experiment !== hoveredExperiment)
+                .on('mouseenter', (event, d) => {
+                  hoveredExperiment = d.experiment;
+
+                  // Show value label on bar
+                  gLabels.selectAll('text').remove();
+                  gLabels.append('text')
+                    .attr('x', d.x + d.width / 2)
+                    .attr('y', d.y - 5)
+                    .attr('text-anchor', 'middle')
+                    .attr('fill', 'var(--text-color)')
+                    .attr('font-size', '11px')
+                    .attr('font-weight', '600')
+                    .text(metric.format(d.mean));
+
+                  updateAll();
+                  tooltip
+                    .style('opacity', 1)
+                    .html(`
+                      <div><strong>${d.experiment}</strong></div>
+                      <div style="margin-top: 4px;">${metric.label}</div>
+                      <div style="margin-top: 4px;"><strong>Mean:</strong> ${metric.format(d.mean)}</div>
+                      <div><strong>Std:</strong> ${metric.format(d.std)}</div>
+                    `);
+                })
+                .on('mousemove', (event) => {
+                  const [mx, my] = d3.pointer(event, container);
+                  tooltip.style('transform', `translate(${mx + 10}px, ${my + 10}px)`);
+                })
+                .on('mouseleave', () => {
+                  hoveredExperiment = null;
+                  gLabels.selectAll('text').remove();
+                  updateAll();
+                  tooltip.style('opacity', 0).style('transform', 'translate(-9999px, -9999px)');
+                });
+
+              // Error bars
+              gErrorBars.selectAll('line')
+                .data(bars)
+                .join('line')
+                .attr('x1', d => d.x + d.width / 2)
+                .attr('x2', d => d.x + d.width / 2)
+                .attr('y1', d => y(d.mean + d.std))
+                .attr('y2', d => y(Math.max(0, d.mean - d.std)))
+                .attr('stroke', '#666')
+                .attr('stroke-width', 1.5)
+                .attr('opacity', 0.6);
+
+              // Error bar caps
+              gErrorBars.selectAll('.cap-top')
+                .data(bars)
+                .join('line')
+                .attr('class', 'cap-top')
+                .attr('x1', d => d.x + d.width / 2 - 3)
+                .attr('x2', d => d.x + d.width / 2 + 3)
+                .attr('y1', d => y(d.mean + d.std))
+                .attr('y2', d => y(d.mean + d.std))
+                .attr('stroke', '#666')
+                .attr('stroke-width', 1.5)
+                .attr('opacity', 0.6);
+
+              gErrorBars.selectAll('.cap-bottom')
+                .data(bars)
+                .join('line')
+                .attr('class', 'cap-bottom')
+                .attr('x1', d => d.x + d.width / 2 - 3)
+                .attr('x2', d => d.x + d.width / 2 + 3)
+                .attr('y1', d => y(Math.max(0, d.mean - d.std)))
+                .attr('y2', d => y(Math.max(0, d.mean - d.std)))
+                .attr('stroke', '#666')
+                .attr('stroke-width', 1.5)
+                .attr('opacity', 0.6);
+            };
+          });
+
+          const updateAll = () => {
+            gridContainer.querySelectorAll('.subplot').forEach(subplot => {
+              if (subplot._render) subplot._render();
+            });
+
+          };
+
+          updateAll();
+
+          if (window.ResizeObserver) {
+            const ro = new ResizeObserver(() => updateAll());
+            ro.observe(container);
+          } else {
+            window.addEventListener('resize', updateAll);
+          }
+        })
+        .catch(err => {
+          container.innerHTML = `<div style="color: red; padding: 20px;">Error: ${err.message}</div>`;
+        });
+    };
+
+    if (document.readyState === 'loading') {
+      document.addEventListener('DOMContentLoaded', () => ensureD3(bootstrap), { once: true });
+    } else {
+      ensureD3(bootstrap);
+    }
+  })();
+</script>

app/src/content/embeds/d3-evaluation2-clamp.html

@@ -0,0 +1,414 @@
+<div class="d3-eval-grid d3-eval-grid-2"></div>
+<style>
+  .d3-eval-grid {
+    padding: 2px;
+    font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
+  }
+
+  .d3-eval-grid .grid-container {
+    display: grid;
+    grid-template-columns: repeat(2, 1fr);
+    gap: 8px;
+  }
+
+  @media (max-width: 768px) {
+    .d3-eval-grid .grid-container {
+      grid-template-columns: 1fr;
+    }
+  }
+
+  .d3-eval-grid .subplot {
+    padding: 4px;
+  }
+
+  .d3-eval-grid .subplot-title {
+    font-size: 12px;
+    font-weight: 600;
+    color: var(--text-color);
+    margin-bottom: 4px;
+    text-align: center;
+  }
+
+
+  .d3-eval-grid .axes path,
+  .d3-eval-grid .axes line {
+    stroke: var(--axis-color);
+  }
+
+  .d3-eval-grid .axes text {
+    fill: var(--tick-color);
+    font-size: 9px;
+  }
+
+  .d3-eval-grid .grid line {
+    stroke: var(--grid-color);
+    stroke-dasharray: 2,2;
+    opacity: 0.5;
+  }
+
+  .d3-eval-grid .axis-label {
+    fill: var(--text-color);
+    font-size: 11px;
+    font-weight: 600;
+  }
+
+  .d3-eval-grid .d3-tooltip {
+    position: absolute;
+    pointer-events: none;
+    padding: 8px 10px;
+    background: var(--surface-bg);
+    border: 1px solid var(--border-color);
+    border-radius: 8px;
+    font-size: 11px;
+    line-height: 1.5;
+    box-shadow: 0 4px 24px rgba(0,0,0,.18);
+    opacity: 0;
+    transition: opacity 0.2s;
+    z-index: 1000;
+  }
+
+  .d3-eval-grid .bar {
+    transition: opacity 0.2s;
+  }
+
+  .d3-eval-grid .bar.dimmed {
+    opacity: 0.2;
+  }
+</style>
+<script>
+  (() => {
+    const ensureD3 = (cb) => {
+      if (window.d3 && typeof window.d3.select === 'function') return cb();
+      let s = document.getElementById('d3-cdn-script');
+      if (!s) {
+        s = document.createElement('script');
+        s.id = 'd3-cdn-script';
+        s.src = 'https://cdn.jsdelivr.net/npm/d3@7/dist/d3.min.js';
+        document.head.appendChild(s);
+      }
+      s.addEventListener('load', () => {
+        if (window.d3 && typeof window.d3.select === 'function') cb();
+      }, { once: true });
+    };
+
+    const bootstrap = () => {
+      const scriptEl = document.currentScript;
+      let container = scriptEl ? scriptEl.previousElementSibling : null;
+      if (!(container && container.classList && container.classList.contains('d3-eval-grid-2'))) {
+        const candidates = Array.from(document.querySelectorAll('.d3-eval-grid-2'))
+          .filter((el) => !(el.dataset && el.dataset.mounted === 'true'));
+        container = candidates[candidates.length - 1] || null;
+      }
+      if (!container) return;
+      if (container.dataset) {
+        if (container.dataset.mounted === 'true') return;
+        container.dataset.mounted = 'true';
+      }
+
+      // Find data attribute
+      let mountEl = container;
+      while (mountEl && !mountEl.getAttribute?.('data-datafiles')) {
+        mountEl = mountEl.parentElement;
+      }
+      let providedData = null;
+      try {
+        const attr = mountEl && mountEl.getAttribute ? mountEl.getAttribute('data-datafiles') : null;
+        if (attr && attr.trim()) {
+          providedData = attr.trim().startsWith('[') ? JSON.parse(attr) : attr.trim();
+        }
+      } catch(_) {}
+
+      // Check for experiments filter attribute
+      let experimentsFilter = null;
+      try {
+        const expAttr = container.getAttribute('data-experiments');
+        if (expAttr) {
+          experimentsFilter = JSON.parse(expAttr);
+        }
+      } catch(_) {}
+
+      const DEFAULT_JSON = '/data/evaluation_summary.json';
+      const ensureDataPrefix = (p) => (typeof p === 'string' && p && !p.includes('/')) ? `/data/${p}` : p;
+
+      const JSON_PATHS = typeof providedData === 'string'
+        ? [ensureDataPrefix(providedData)]
+        : [
+            DEFAULT_JSON,
+            './assets/data/evaluation_summary.json',
+            '../assets/data/evaluation_summary.json',
+            '../../assets/data/evaluation_summary.json'
+          ];
+
+      const fetchFirstAvailable = async (paths) => {
+        for (const p of paths) {
+          try {
+            const r = await fetch(p, { cache: 'no-cache' });
+            if (r.ok) return await r.json();
+          } catch(_){}
+        }
+        throw new Error('JSON not found');
+      };
+
+      fetchFirstAvailable(JSON_PATHS)
+        .then(rawData => {
+          // Chart 2: Add clamping experiments (but reserve space for all)
+          const allExperiments = ['Prompt', 'Basic steering', 'Clamping', 'Clamping + Penalty', '2D optimized', '8D optimized'];
+          const visibleExperiments = ['Prompt', 'Basic steering', 'Clamping', 'Clamping + Penalty'];
+
+          // Metrics in 2x4 grid layout (8 metrics)
+          const metrics = [
+            { key: 'llm_score_concept', label: 'LLM Concept Score', format: d3.format('.2f') },
+            { key: 'eiffel', label: 'Explicit Concept Presence', format: d3.format('.2f') },
+            { key: 'llm_score_instruction', label: 'LLM Instruction Score', format: d3.format('.2f') },
+            { key: 'minus_log_prob', label: 'Surprise in Original Model', format: d3.format('.2f') },
+            { key: 'llm_score_fluency', label: 'LLM Fluency Score', format: d3.format('.2f') },
+            { key: 'rep3', label: '3-gram Repetition Fraction', format: d3.format('.2f') },
+            { key: 'mean_llm_score', label: 'Mean LLM Score', format: d3.format('.2f') },
+            { key: 'harmonic_llm_score', label: 'Harmonic Mean LLM Score', format: d3.format('.2f') }
+          ];
+
+          // Restructure data
+          const data = {};
+          rawData.forEach(d => {
+            if (!data[d.metric]) data[d.metric] = {};
+            data[d.metric][d.experiment] = { mean: d.mean, std: d.std };
+          });
+
+          // Color palette - consistent across all charts
+          const allColors = {
+            'Prompt': '#4c4c4c',
+            'Basic steering': '#b2b2b2',
+            'Clamping': '#b2b2cc',
+            'Clamping + Penalty': '#b2b2e6',
+            '2D optimized': '#b2ffb2',
+            '8D optimized': '#ffb2ff'
+          };
+
+          const gridContainer = document.createElement('div');
+          gridContainer.className = 'grid-container';
+          container.appendChild(gridContainer);
+
+          // Tooltip
+          const tooltip = d3.select(container).append('div')
+            .attr('class', 'd3-tooltip')
+            .style('transform', 'translate(-9999px, -9999px)');
+
+          let hoveredExperiment = null;
+
+          // Create each subplot
+          metrics.forEach((metric, idx) => {
+            const subplot = document.createElement('div');
+            subplot.className = 'subplot';
+            subplot.dataset.metric = metric.key;
+            gridContainer.appendChild(subplot);
+
+            const title = document.createElement('div');
+            title.className = 'subplot-title';
+            title.textContent = metric.label;
+            subplot.appendChild(title);
+
+            const svg = d3.select(subplot).append('svg')
+              .attr('width', '100%')
+              .style('display', 'block');
+
+            const g = svg.append('g');
+            const gGrid = g.append('g').attr('class', 'grid');
+            const gBars = g.append('g').attr('class', 'bars');
+            const gErrorBars = g.append('g').attr('class', 'error-bars');
+            const gAxes = g.append('g').attr('class', 'axes');
+            const gLabels = g.append('g').attr('class', 'value-labels');
+
+            subplot._render = () => {
+              const width = subplot.clientWidth || 300;
+              const height = Math.max(200, Math.round(width * 0.6));
+              const margin = { top: 10, right: 20, bottom: 70, left: 42 };
+              const innerWidth = width - margin.left - margin.right;
+              const innerHeight = height - margin.top - margin.bottom;
+
+              svg.attr('height', height);
+              g.attr('transform', `translate(${margin.left},${margin.top})`);
+
+              // Scales - use all experiments for consistent positioning
+              const x = d3.scaleBand()
+                .domain(allExperiments)
+                .range([0, innerWidth])
+                .padding(0.2);
+
+              // Fixed y-axis ranges based on metric type
+              const yDomains = {
+                'llm_score_concept': [0, 2],
+                'llm_score_instruction': [0, 2],
+                'llm_score_fluency': [0, 2],
+                'mean_llm_score': [0, 2],
+                'harmonic_llm_score': [0, 2],
+                'eiffel': [0, 1],
+                'minus_log_prob': [0, 2],
+                'rep3': [0, 0.5]
+              };
+
+              const y = d3.scaleLinear()
+                .domain(yDomains[metric.key] || [0, 1])
+                .range([innerHeight, 0]);
+
+              // Grid
+              gGrid.selectAll('*').remove();
+              gGrid.selectAll('line')
+                .data(y.ticks(4))
+                .join('line')
+                .attr('x1', 0)
+                .attr('x2', innerWidth)
+                .attr('y1', d => y(d))
+                .attr('y2', d => y(d));
+
+              // Axes
+              gAxes.selectAll('*').remove();
+
+              const xAxis = gAxes.append('g')
+                .attr('transform', `translate(0,${innerHeight})`)
+                .call(d3.axisBottom(x).tickSize(3));
+
+              // Only show labels for visible experiments
+              xAxis.selectAll('text')
+                .attr('transform', 'rotate(-45)')
+                .style('text-anchor', 'end')
+                .attr('dx', '-0.5em')
+                .attr('dy', '0.15em')
+                .style('opacity', function() {
+                  const text = d3.select(this).text();
+                  return visibleExperiments.includes(text) ? 1 : 0;
+                });
+
+              gAxes.append('g')
+                .call(d3.axisLeft(y).ticks(4).tickFormat(metric.format).tickSize(3));
+
+              // Draw bars (only for visible experiments)
+              const bars = [];
+              visibleExperiments.forEach(exp => {
+                const d = data[metric.key]?.[exp];
+                if (d) {
+                  bars.push({
+                    experiment: exp,
+                    mean: d.mean,
+                    std: d.std,
+                    color: allColors[exp],
+                    x: x(exp),
+                    y: y(d.mean),
+                    width: x.bandwidth(),
+                    height: innerHeight - y(d.mean)
+                  });
+                }
+              });
+
+              gBars.selectAll('rect')
+                .data(bars)
+                .join('rect')
+                .attr('class', 'bar')
+                .attr('x', d => d.x)
+                .attr('y', d => d.y)
+                .attr('width', d => d.width)
+                .attr('height', d => d.height)
+                .attr('fill', d => d.color)
+                .attr('rx', 2)
+                .classed('dimmed', d => hoveredExperiment && d.experiment !== hoveredExperiment)
+                .on('mouseenter', (event, d) => {
+                  hoveredExperiment = d.experiment;
+
+                  // Show value label on bar
+                  gLabels.selectAll('text').remove();
+                  gLabels.append('text')
+                    .attr('x', d.x + d.width / 2)
+                    .attr('y', d.y - 5)
+                    .attr('text-anchor', 'middle')
+                    .attr('fill', 'var(--text-color)')
+                    .attr('font-size', '11px')
+                    .attr('font-weight', '600')
+                    .text(metric.format(d.mean));
+
+                  updateAll();
+                  tooltip
+                    .style('opacity', 1)
+                    .html(`
+                      <div><strong>${d.experiment}</strong></div>
+                      <div style="margin-top: 4px;">${metric.label}</div>
+                      <div style="margin-top: 4px;"><strong>Mean:</strong> ${metric.format(d.mean)}</div>
+                      <div><strong>Std:</strong> ${metric.format(d.std)}</div>
+                    `);
+                })
+                .on('mousemove', (event) => {
+                  const [mx, my] = d3.pointer(event, container);
+                  tooltip.style('transform', `translate(${mx + 10}px, ${my + 10}px)`);
+                })
+                .on('mouseleave', () => {
+                  hoveredExperiment = null;
+                  gLabels.selectAll('text').remove();
+                  updateAll();
+                  tooltip.style('opacity', 0).style('transform', 'translate(-9999px, -9999px)');
+                });
+
+              // Error bars
+              gErrorBars.selectAll('line')
+                .data(bars)
+                .join('line')
+                .attr('x1', d => d.x + d.width / 2)
+                .attr('x2', d => d.x + d.width / 2)
+                .attr('y1', d => y(d.mean + d.std))
+                .attr('y2', d => y(Math.max(0, d.mean - d.std)))
+                .attr('stroke', '#666')
+                .attr('stroke-width', 1.5)
+                .attr('opacity', 0.6);
+
+              // Error bar caps
+              gErrorBars.selectAll('.cap-top')
+                .data(bars)
+                .join('line')
+                .attr('class', 'cap-top')
+                .attr('x1', d => d.x + d.width / 2 - 3)
+                .attr('x2', d => d.x + d.width / 2 + 3)
+                .attr('y1', d => y(d.mean + d.std))
+                .attr('y2', d => y(d.mean + d.std))
+                .attr('stroke', '#666')
+                .attr('stroke-width', 1.5)
+                .attr('opacity', 0.6);
+
+              gErrorBars.selectAll('.cap-bottom')
+                .data(bars)
+                .join('line')
+                .attr('class', 'cap-bottom')
+                .attr('x1', d => d.x + d.width / 2 - 3)
+                .attr('x2', d => d.x + d.width / 2 + 3)
+                .attr('y1', d => y(Math.max(0, d.mean - d.std)))
+                .attr('y2', d => y(Math.max(0, d.mean - d.std)))
+                .attr('stroke', '#666')
+                .attr('stroke-width', 1.5)
+                .attr('opacity', 0.6);
+            };
+          });
+
+          const updateAll = () => {
+            gridContainer.querySelectorAll('.subplot').forEach(subplot => {
+              if (subplot._render) subplot._render();
+            });
+
+          };
+
+          updateAll();
+
+          if (window.ResizeObserver) {
+            const ro = new ResizeObserver(() => updateAll());
+            ro.observe(container);
+          } else {
+            window.addEventListener('resize', updateAll);
+          }
+        })
+        .catch(err => {
+          container.innerHTML = `<div style="color: red; padding: 20px;">Error: ${err.message}</div>`;
+        });
+    };
+
+    if (document.readyState === 'loading') {
+      document.addEventListener('DOMContentLoaded', () => ensureD3(bootstrap), { once: true });
+    } else {
+      ensureD3(bootstrap);
+    }
+  })();
+</script>

app/src/content/embeds/d3-evaluation3-multi.html

@@ -0,0 +1,414 @@
+<div class="d3-eval-grid d3-eval-grid-3"></div>
+<style>
+  .d3-eval-grid {
+    padding: 2px;
+    font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
+  }
+
+  .d3-eval-grid .grid-container {
+    display: grid;
+    grid-template-columns: repeat(2, 1fr);
+    gap: 8px;
+  }
+
+  @media (max-width: 768px) {
+    .d3-eval-grid .grid-container {
+      grid-template-columns: 1fr;
+    }
+  }
+
+  .d3-eval-grid .subplot {
+    padding: 4px;
+  }
+
+  .d3-eval-grid .subplot-title {
+    font-size: 12px;
+    font-weight: 600;
+    color: var(--text-color);
+    margin-bottom: 4px;
+    text-align: center;
+  }
+
+
+  .d3-eval-grid .axes path,
+  .d3-eval-grid .axes line {
+    stroke: var(--axis-color);
+  }
+
+  .d3-eval-grid .axes text {
+    fill: var(--tick-color);
+    font-size: 9px;
+  }
+
+  .d3-eval-grid .grid line {
+    stroke: var(--grid-color);
+    stroke-dasharray: 2,2;
+    opacity: 0.5;
+  }
+
+  .d3-eval-grid .axis-label {
+    fill: var(--text-color);
+    font-size: 11px;
+    font-weight: 600;
+  }
+
+  .d3-eval-grid .d3-tooltip {
+    position: absolute;
+    pointer-events: none;
+    padding: 8px 10px;
+    background: var(--surface-bg);
+    border: 1px solid var(--border-color);
+    border-radius: 8px;
+    font-size: 11px;
+    line-height: 1.5;
+    box-shadow: 0 4px 24px rgba(0,0,0,.18);
+    opacity: 0;
+    transition: opacity 0.2s;
+    z-index: 1000;
+  }
+
+  .d3-eval-grid .bar {
+    transition: opacity 0.2s;
+  }
+
+  .d3-eval-grid .bar.dimmed {
+    opacity: 0.2;
+  }
+</style>
+<script>
+  (() => {
+    const ensureD3 = (cb) => {
+      if (window.d3 && typeof window.d3.select === 'function') return cb();
+      let s = document.getElementById('d3-cdn-script');
+      if (!s) {
+        s = document.createElement('script');
+        s.id = 'd3-cdn-script';
+        s.src = 'https://cdn.jsdelivr.net/npm/d3@7/dist/d3.min.js';
+        document.head.appendChild(s);
+      }
+      s.addEventListener('load', () => {
+        if (window.d3 && typeof window.d3.select === 'function') cb();
+      }, { once: true });
+    };
+
+    const bootstrap = () => {
+      const scriptEl = document.currentScript;
+      let container = scriptEl ? scriptEl.previousElementSibling : null;
+      if (!(container && container.classList && container.classList.contains('d3-eval-grid-3'))) {
+        const candidates = Array.from(document.querySelectorAll('.d3-eval-grid-3'))
+          .filter((el) => !(el.dataset && el.dataset.mounted === 'true'));
+        container = candidates[candidates.length - 1] || null;
+      }
+      if (!container) return;
+      if (container.dataset) {
+        if (container.dataset.mounted === 'true') return;
+        container.dataset.mounted = 'true';
+      }
+
+      // Find data attribute
+      let mountEl = container;
+      while (mountEl && !mountEl.getAttribute?.('data-datafiles')) {
+        mountEl = mountEl.parentElement;
+      }
+      let providedData = null;
+      try {
+        const attr = mountEl && mountEl.getAttribute ? mountEl.getAttribute('data-datafiles') : null;
+        if (attr && attr.trim()) {
+          providedData = attr.trim().startsWith('[') ? JSON.parse(attr) : attr.trim();
+        }
+      } catch(_) {}
+
+      // Check for experiments filter attribute
+      let experimentsFilter = null;
+      try {
+        const expAttr = container.getAttribute('data-experiments');
+        if (expAttr) {
+          experimentsFilter = JSON.parse(expAttr);
+        }
+      } catch(_) {}
+
+      const DEFAULT_JSON = '/data/evaluation_summary.json';
+      const ensureDataPrefix = (p) => (typeof p === 'string' && p && !p.includes('/')) ? `/data/${p}` : p;
+
+      const JSON_PATHS = typeof providedData === 'string'
+        ? [ensureDataPrefix(providedData)]
+        : [
+            DEFAULT_JSON,
+            './assets/data/evaluation_summary.json',
+            '../assets/data/evaluation_summary.json',
+            '../../assets/data/evaluation_summary.json'
+          ];
+
+      const fetchFirstAvailable = async (paths) => {
+        for (const p of paths) {
+          try {
+            const r = await fetch(p, { cache: 'no-cache' });
+            if (r.ok) return await r.json();
+          } catch(_){}
+        }
+        throw new Error('JSON not found');
+      };
+
+      fetchFirstAvailable(JSON_PATHS)
+        .then(rawData => {
+          // Chart 3: All experiments including multi-layer optimization
+          const allExperiments = ['Prompt', 'Basic steering', 'Clamping', 'Clamping + Penalty', '2D optimized', '8D optimized'];
+          const visibleExperiments = allExperiments;
+
+          // Metrics in 2x4 grid layout (8 metrics)
+          const metrics = [
+            { key: 'llm_score_concept', label: 'LLM Concept Score', format: d3.format('.2f') },
+            { key: 'eiffel', label: 'Explicit Concept Presence', format: d3.format('.2f') },
+            { key: 'llm_score_instruction', label: 'LLM Instruction Score', format: d3.format('.2f') },
+            { key: 'minus_log_prob', label: 'Surprise in Original Model', format: d3.format('.2f') },
+            { key: 'llm_score_fluency', label: 'LLM Fluency Score', format: d3.format('.2f') },
+            { key: 'rep3', label: '3-gram Repetition Fraction', format: d3.format('.2f') },
+            { key: 'mean_llm_score', label: 'Mean LLM Score', format: d3.format('.2f') },
+            { key: 'harmonic_llm_score', label: 'Harmonic Mean LLM Score', format: d3.format('.2f') }
+          ];
+
+          // Restructure data
+          const data = {};
+          rawData.forEach(d => {
+            if (!data[d.metric]) data[d.metric] = {};
+            data[d.metric][d.experiment] = { mean: d.mean, std: d.std };
+          });
+
+          // Color palette - consistent across all charts
+          const allColors = {
+            'Prompt': '#4c4c4c',
+            'Basic steering': '#b2b2b2',
+            'Clamping': '#b2b2cc',
+            'Clamping + Penalty': '#b2b2e6',
+            '2D optimized': '#b2ffb2',
+            '8D optimized': '#ffb2ff'
+          };
+
+          const gridContainer = document.createElement('div');
+          gridContainer.className = 'grid-container';
+          container.appendChild(gridContainer);
+
+          // Tooltip
+          const tooltip = d3.select(container).append('div')
+            .attr('class', 'd3-tooltip')
+            .style('transform', 'translate(-9999px, -9999px)');
+
+          let hoveredExperiment = null;
+
+          // Create each subplot
+          metrics.forEach((metric, idx) => {
+            const subplot = document.createElement('div');
+            subplot.className = 'subplot';
+            subplot.dataset.metric = metric.key;
+            gridContainer.appendChild(subplot);
+
+            const title = document.createElement('div');
+            title.className = 'subplot-title';
+            title.textContent = metric.label;
+            subplot.appendChild(title);
+
+            const svg = d3.select(subplot).append('svg')
+              .attr('width', '100%')
+              .style('display', 'block');
+
+            const g = svg.append('g');
+            const gGrid = g.append('g').attr('class', 'grid');
+            const gBars = g.append('g').attr('class', 'bars');
+            const gErrorBars = g.append('g').attr('class', 'error-bars');
+            const gAxes = g.append('g').attr('class', 'axes');
+            const gLabels = g.append('g').attr('class', 'value-labels');
+
+            subplot._render = () => {
+              const width = subplot.clientWidth || 300;
+              const height = Math.max(200, Math.round(width * 0.6));
+              const margin = { top: 10, right: 20, bottom: 70, left: 42 };
+              const innerWidth = width - margin.left - margin.right;
+              const innerHeight = height - margin.top - margin.bottom;
+
+              svg.attr('height', height);
+              g.attr('transform', `translate(${margin.left},${margin.top})`);
+
+              // Scales - use all experiments for consistent positioning
+              const x = d3.scaleBand()
+                .domain(allExperiments)
+                .range([0, innerWidth])
+                .padding(0.2);
+
+              // Fixed y-axis ranges based on metric type
+              const yDomains = {
+                'llm_score_concept': [0, 2],
+                'llm_score_instruction': [0, 2],
+                'llm_score_fluency': [0, 2],
+                'mean_llm_score': [0, 2],
+                'harmonic_llm_score': [0, 2],
+                'eiffel': [0, 1],
+                'minus_log_prob': [0, 2],
+                'rep3': [0, 0.5]
+              };
+
+              const y = d3.scaleLinear()
+                .domain(yDomains[metric.key] || [0, 1])
+                .range([innerHeight, 0]);
+
+              // Grid
+              gGrid.selectAll('*').remove();
+              gGrid.selectAll('line')
+                .data(y.ticks(4))
+                .join('line')
+                .attr('x1', 0)
+                .attr('x2', innerWidth)
+                .attr('y1', d => y(d))
+                .attr('y2', d => y(d));
+
+              // Axes
+              gAxes.selectAll('*').remove();
+
+              const xAxis = gAxes.append('g')
+                .attr('transform', `translate(0,${innerHeight})`)
+                .call(d3.axisBottom(x).tickSize(3));
+
+              // Only show labels for visible experiments
+              xAxis.selectAll('text')
+                .attr('transform', 'rotate(-45)')
+                .style('text-anchor', 'end')
+                .attr('dx', '-0.5em')
+                .attr('dy', '0.15em')
+                .style('opacity', function() {
+                  const text = d3.select(this).text();
+                  return visibleExperiments.includes(text) ? 1 : 0;
+                });
+
+              gAxes.append('g')
+                .call(d3.axisLeft(y).ticks(4).tickFormat(metric.format).tickSize(3));
+
+              // Draw bars (only for visible experiments)
+              const bars = [];
+              visibleExperiments.forEach(exp => {
+                const d = data[metric.key]?.[exp];
+                if (d) {
+                  bars.push({
+                    experiment: exp,
+                    mean: d.mean,
+                    std: d.std,
+                    color: allColors[exp],
+                    x: x(exp),
+                    y: y(d.mean),
+                    width: x.bandwidth(),
+                    height: innerHeight - y(d.mean)
+                  });
+                }
+              });
+
+              gBars.selectAll('rect')
+                .data(bars)
+                .join('rect')
+                .attr('class', 'bar')
+                .attr('x', d => d.x)
+                .attr('y', d => d.y)
+                .attr('width', d => d.width)
+                .attr('height', d => d.height)
+                .attr('fill', d => d.color)
+                .attr('rx', 2)
+                .classed('dimmed', d => hoveredExperiment && d.experiment !== hoveredExperiment)
+                .on('mouseenter', (event, d) => {
+                  hoveredExperiment = d.experiment;
+
+                  // Show value label on bar
+                  gLabels.selectAll('text').remove();
+                  gLabels.append('text')
+                    .attr('x', d.x + d.width / 2)
+                    .attr('y', d.y - 5)
+                    .attr('text-anchor', 'middle')
+                    .attr('fill', 'var(--text-color)')
+                    .attr('font-size', '11px')
+                    .attr('font-weight', '600')
+                    .text(metric.format(d.mean));
+
+                  updateAll();
+                  tooltip
+                    .style('opacity', 1)
+                    .html(`
+                      <div><strong>${d.experiment}</strong></div>
+                      <div style="margin-top: 4px;">${metric.label}</div>
+                      <div style="margin-top: 4px;"><strong>Mean:</strong> ${metric.format(d.mean)}</div>
+                      <div><strong>Std:</strong> ${metric.format(d.std)}</div>
+                    `);
+                })
+                .on('mousemove', (event) => {
+                  const [mx, my] = d3.pointer(event, container);
+                  tooltip.style('transform', `translate(${mx + 10}px, ${my + 10}px)`);
+                })
+                .on('mouseleave', () => {
+                  hoveredExperiment = null;
+                  gLabels.selectAll('text').remove();
+                  updateAll();
+                  tooltip.style('opacity', 0).style('transform', 'translate(-9999px, -9999px)');
+                });
+
+              // Error bars
+              gErrorBars.selectAll('line')
+                .data(bars)
+                .join('line')
+                .attr('x1', d => d.x + d.width / 2)
+                .attr('x2', d => d.x + d.width / 2)
+                .attr('y1', d => y(d.mean + d.std))
+                .attr('y2', d => y(Math.max(0, d.mean - d.std)))
+                .attr('stroke', '#666')
+                .attr('stroke-width', 1.5)
+                .attr('opacity', 0.6);
+
+              // Error bar caps
+              gErrorBars.selectAll('.cap-top')
+                .data(bars)
+                .join('line')
+                .attr('class', 'cap-top')
+                .attr('x1', d => d.x + d.width / 2 - 3)
+                .attr('x2', d => d.x + d.width / 2 + 3)
+                .attr('y1', d => y(d.mean + d.std))
+                .attr('y2', d => y(d.mean + d.std))
+                .attr('stroke', '#666')
+                .attr('stroke-width', 1.5)
+                .attr('opacity', 0.6);
+
+              gErrorBars.selectAll('.cap-bottom')
+                .data(bars)
+                .join('line')
+                .attr('class', 'cap-bottom')
+                .attr('x1', d => d.x + d.width / 2 - 3)
+                .attr('x2', d => d.x + d.width / 2 + 3)
+                .attr('y1', d => y(Math.max(0, d.mean - d.std)))
+                .attr('y2', d => y(Math.max(0, d.mean - d.std)))
+                .attr('stroke', '#666')
+                .attr('stroke-width', 1.5)
+                .attr('opacity', 0.6);
+            };
+          });
+
+          const updateAll = () => {
+            gridContainer.querySelectorAll('.subplot').forEach(subplot => {
+              if (subplot._render) subplot._render();
+            });
+
+          };
+
+          updateAll();
+
+          if (window.ResizeObserver) {
+            const ro = new ResizeObserver(() => updateAll());
+            ro.observe(container);
+          } else {
+            window.addEventListener('resize', updateAll);
+          }
+        })
+        .catch(err => {
+          container.innerHTML = `<div style="color: red; padding: 20px;">Error: ${err.message}</div>`;
+        });
+    };
+
+    if (document.readyState === 'loading') {
+      document.addEventListener('DOMContentLoaded', () => ensureD3(bootstrap), { once: true });
+    } else {
+      ensureD3(bootstrap);
+    }
+  })();
+</script>