Noy Sternlicht

I'm a second-year PhD student at the Hebrew University of Jerusalem, advised by Dr. Tom Hope. My research leverages natural language processing to analyze scientific literature and identify new research directions.

I'm also an ex-Amazonian, where I worked as a software engineer for two years.

When I’m not wrangling papers, you can find me at my sewing machine 🧵🪡

Research & Publications

Old Ideas, Novel Problems: The Instability of LLM-Based Novelty Evaluation

Noy Sternlicht, Simra Shahid, Peter Jansen, Daniel S. Weld, Pao Siangliulue, Tom Hope

Arxiv Preprint

Automated ideation systems are often evaluated on the novelty of their ideas, a judgment increasingly delegated to LLMs. So, how do novelty judges perform? Not well. We present a systematic controlled study of novelty evaluation design choices. We find that small changes to the prompt have large consequences, retrieval and larger reasoning budgets help little, and purpose-built novelty evaluators are outperformed by our cheapest prompted baseline. These results raise questions about reported novelty gains of automated ideation systems.

LLM-as-a-Judge Novelty Evaluation Ideation NLP for Science Evaluation
CHIMERA: A Knowledge Base of Idea Recombination in Scientific Literature

Noy Sternlicht, Tom Hope

ACL 2026 (main)

We present CHIMERA, a large-scale knowledge base of over 28K idea recombination examples mined from scientific literature. CHIMERA supports large-scale analysis of scientific creativity and enables applications in hypothesis generation and cross-domain exploration. We demonstrate how this data can be used to train models that generate novel and inspiring research directions.

Literature Mining Ideation Information Extraction NLP for Science Knowledge Graph Meta Science
In-depth Research Impact Summarization through Fine-Grained Temporal Citation Analysis

Hiba Arnaout, Noy Sternlicht, Tom Hope, Iryna Gurevych

ACL 2026 (main)

We introduce a new way to understand the impact of scientific papers—beyond citation counts—by generating nuanced, time-aware summaries that reflect both praise and critique from later research.

Citation Analysis Research Impact Summarization NLP for Science Reference-Free Evaluation
Debatable Intelligence: Benchmarking LLM Judges via Debate Speech Evaluation

Noy Sternlicht, Ariel Gera, Roy Bar-Haim, Tom Hope, Noam Slonim

EMNLP 2025 (main)

We assess the judgment capabilities and behavior of LLMs by analyzing how they rate debate speeches - long texts that argue for or against a controversial topic. Our analysis reveals key differences between model and human judgments. We also investigate the ability of frontier LLMs to generate persuasive, opinionated speeches, showing that models may perform at a human level on this task.

LLM-as-a-Judge Language Model Benchmarking Debate Evaluation