Florence-VL: Enhancing Vision-Language Models with Generative Vision Encoder and Depth-Breadth Fusion ckpt 8B: huggingface.co/jiuhai/flore... demo: huggingface.co/spaces/jiuha... code: github.com/JiuhaiChen/F... paper: arxiv.org/abs/2412.04424

NVILA, a VLM, enhances VILA by scaling spatial and temporal resolutions before compressing visual tokens, enabling efficient high-resolution image & long video processing. Cuts training costs by 4.5X, improves memory & latency, and outperforms top VLMs on benchmarks. Code & models will be released 🔜

ClearerVoice-Studio by Tongyi Lab is a versatile voice processing framework offering noise removal, speech separation, audio-video speaker extraction, and tools for model fine-tuning and optimization. git: github.com/modelscope/C... demo: huggingface.co/spaces/aliba...

PaliGemma 2: A Family of Versatile VLMs for Transfer paper: arxiv.org/abs/2412.03555

Home robotics just got a boost. Stretch AI - a new open-source suite of tools, tutorials, and reference code to explore and build AI-enabled home robot applications.

Liquid AI introduces synthesis of tailored architectures (STAR) a new approach to automate neural network design tailored to various tasks and hardware setups. 🔗: www.liquid.ai/research/aut...

JetFormer: An Autoregressive Generative Model of Raw Images and Text paper: arxiv.org/abs/2411.19722 JetFormer unifies text and image modeling with a normalizing flow, enabling strong text-to-image generation and image understanding.

Marconi: Prefix Caching for the Era of Hybrid LLMs paper: arxiv.org/abs/2411.19379 Marconi improves caching for hybrid LLMs with policies optimizing reuse likelihood and compute savings, achieving 34.4× higher token hit rates and significantly reducing latency.

DeMo: Decoupled Momentum Optimization code: github.com/bloc97/DeMo paper: arxiv.org/abs/2411.19870

Lift3D Foundation Policy: Lifting 2D Large-Scale Pretrained Models for Robust 3D Robotic Manipulation code: github.com/PKU-HMI-Lab/... paper: arxiv.org/abs/2411.18623 project: lift3d-web.github.io

Efficient Track Anything paper: arxiv.org/abs/2411.18933 project page: yformer.github.io/efficient-tr...

OMuleT: Orchestrating Multiple Tools for Practicable Conversational Recommendation Roblox paper: arxiv.org/abs/2411.19352 A CRS enhancing LLMs with 10+ tools improves recommendations and shares insights from design, evaluation, and deployment.

Training Agents with Weakly Supervised Feedback from Large Language Models paper: arxiv.org/abs/2411.19547

Critical Tokens Matter: Token-Level Contrastive Estimation Enhence LLM's Reasoning Capability paper: arxiv.org/abs/2411.19943

Reverse Thinking Makes LLMs Stronger Reasoners paper: arxiv.org/abs/2411.19865 RevThink improves LLM reasoning by 13.53% using structured forward-backward reasoning, ensuring strong generalization and data efficiency.

JetFormer: An Autoregressive Generative Model of Raw Images and Text paper: arxiv.org/abs/2411.19722 JetFormer unifies text and image modeling with a normalizing flow, enabling strong text-to-image generation and image understanding.

Q-learning-based Model-free Safety Filter paper: arxiv.org/abs/2411.19809 A plug-and-play model-free safety filter uses Q-learning to ensure safe actions in robotics, integrating easily with RL algorithms. Simulations and real-world tests confirm its effectiveness.

KV Shifting Attention Enhances Language Modeling paper: arxiv.org/abs/2411.19574 KV shifting attention enhances induction heads in LLMs improving efficiency, in-context learning, and faster convergence, even in models with over 10 billion parameters.

CATP-LLM is a cost-efficient tool planning framework for LLMs, using a planning language for concurrent execution and offline reinforcement learning to balance performance and cost. It outperforms GPT-4 on OpenCATP, with up to 30.2% better performance and 45.8% lower costs.

Directed Token Sliding The Token Sliding problem examines transforming one token configuration into another on a graph by sliding tokens while keeping independence. It is PSPACE-complete for various graph types but solvable in polynomial time for oriented cycles and cographs.

We’re here with good intentions. A lot of the researchers here are genuinely helpful—take the time to follow them and explore their work. Our aim is to contribute, grow, and make things better for everyone.

bluesky datasets: 🔗: zenodo.org/records/1108... 🔗: huggingface.co/datasets/alp... 🔗: huggingface.co/datasets/inf... 🔗: huggingface.co/datasets/not...

Star Attention: Efficient LLM Inference over Long Sequences 🔗: github.com/NVIDIA/Star-... paper: arxiv.org/abs/2411.17116

Skywork-o1-Open

ShowUI: One Vision-Language-Action Model for GUI Visual Agent ShowUI-2B: huggingface.co/showlab/Show... paper: arxiv.org/abs/2411.17465 🔗: github.com/showlab/ShowUI

A Survey on LLM-as-a-Judge paper: arxiv.org/abs/2411.15594 🔗: github.com/IDEA-FinAI/L...

From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge 🔗: github.com/llm-as-a-jud... paper: arxiv.org/abs/2411.16594 project page: llm-as-a-judge.github.io

Enhancing LLM Reasoning via Critique Models with Test-Time and Training-Time Supervision paper: arxiv.org/abs/2411.16579 project page: mathcritique.github.io

TEXGen: a Generative Diffusion Model for Mesh Textures paper: arxiv.org/abs/2411.14740 project page: cvmi-lab.github.io/TEXGen/

NuminaMath dataset now under Apache 2.0 license.

High-Resolution Image Synthesis via Next-Token Prediction D-JEPA⋅T2I extends D-JEPA for high-resolution text-to-image generation, using flow matching loss, multimodal transformers, and Visual Rotary Positional Embedding for efficient learning, achieving sota results with next-token prediction.

Multi-granularity Interest Retrieval and Refinement Network for Long-Term User Behavior Modeling in CTR Prediction paper: arxiv.org/abs/2411.15005 🔗: github.com/psycho-demon...

LIBER: Lifelong User Behavior Modeling Based on Large Language Models LIBER, an LLM-based framework, models lifelong user behavior for recommender systems, improving adaptability and recommendations. Huawei’s deployment boosted play count by 3.01% and play time by 7.69%.

XGrammar: Flexible and Efficient Structured Generation Engine for Large Language Models paper: arxiv.org/abs/2411.15100

DyCoke: Dynamic Compression of Tokens for Fast Video Large Language Models paper: arxiv.org/abs/2411.15024 🔗: github.com/KD-TAO/DyCoke

AttriBoT: A Bag of Tricks for Efficiently Approximating Leave-One-Out Context Attribution paper: arxiv.org/abs/2411.15102 🔗: github.com/r-three/Attr...

REDUCIO! Generating 1024×1024 Video within 16 Seconds using Extremely Compressed Motion Latents paper: arxiv.org/abs/2411.13552

MobileCLIP zero-shot image classification app for iOS 👀 github.com/apple/ml-mob...

Hymba: A Hybrid-head Architecture for Small Language Models paper: arxiv.org/abs/2411.13676 Hymba-1.5B-Base 🔜 huggingface.co/nvidia/Hymba... Hymba-1.5B-Instruct 🔜 huggingface.co/nvidia/Hymba...

Multimodal Autoregressive Pre-training of Large Vision Encoders  : github.com/apple/ml-aim  : arxiv.org/abs/2411.14402  : huggingface.co/collections/...

It seems like some developers are misunderstanding the concept of “intents.” Relying on outdated approaches to create new functionalities is not a solution.

Text-based chatbots hit a wall in user experience. For real progress, we need to move beyond just typing out questions and answers.

SILC: It goes beyond CLIP by adding local-to-global learning. The result? Improved performance across the board—from classification and retrieval to segmentation. SILC: Improving Vision Language Pretraining with Self-Distillation

Want to optimize small models with minimal data? Let's Synthesize Step by Step: Iterative Dataset Synthesis with Large Language Models by Extrapolating Errors from Small Models ↓

Curious about the privacy risks in large language models? Ethicist dives deep to reveal what data these models actually retain. A crucial step in understanding and mitigating data leakage!

EFT technique separates knowledge from pre-training and fine-tuning in language models. Upscaled fine-tuning boosts helpfulness, while upscaled pre-training improves factuality. Allows behavior tweaks at test-time without more training. ↓

Paper studies retrieval in large language models for long-form QA. Finds that evidence selection and multi-document summarization are key. Emphasizes the need to improve retrievers to reduce errors and contradictions. We’re just scratching the surface of what these systems can do