Web Search Plus

Stop choosing search providers. Let the skill do it for you.

This skill connects you to 6 search providers (Serper, Tavily, Exa, Perplexity, You.com, SearXNG) and automatically picks the best one for each query. Shopping question? → Google results. Research question? → Deep research engine. Need a direct answer? → AI-synthesized with citations. Want privacy? → Self-hosted option.

---

✨ What Makes This Different?

• Just search — No need to think about which provider to use
• Smart routing — Analyzes your query and picks the best provider automatically
• 6 providers, 1 interface — Google results, research engines, neural search, AI answers with citations, RAG-optimized, and privacy-first all in one
• Works with just 1 key — Start with any single provider, add more later
• Free options available — SearXNG is completely free (self-hosted)

---

🚀 Quick Start

# Interactive setup (recommended for first run)
python3 scripts/setup.py

# Or manual: copy config and add your keys
cp config.example.json config.json

The wizard explains each provider, collects API keys, and configures defaults.

---

🔑 API Keys

You only need ONE key to get started. Add more providers later for better coverage.

Provider	Free Tier	Best For	Sign Up
Serper	2,500/mo	Shopping, prices, local, news	serper.dev
Tavily	1,000/mo	Research, explanations, academic	tavily.com
Exa	1,000/mo	"Similar to X", startups, papers	exa.ai
Perplexity	Via Kilo	Direct answers with citations	kilo.ai
You.com	Limited	Real-time info, AI/RAG context	api.you.com
SearXNG	FREE ✅	Privacy, multi-source, $0 cost	Self-hosted

Setting your keys:

# Option A: .env file (recommended)
export SERPER_API_KEY="your-key"
export TAVILY_API_KEY="your-key"

# Option B: config.json
{ "serper": { "api_key": "your-key" } }

---

🎯 When to Use Which Provider

I want to...	Provider	Example Query
Find product prices	Serper	"iPhone 16 Pro Max price"
Find restaurants/stores nearby	Serper	"best pizza near me"
Understand how something works	Tavily	"how does HTTPS encryption work"
Do deep research	Tavily	"climate change research 2024"
Find companies like X	Exa	"startups similar to Notion"
Find research papers	Exa	"transformer architecture papers"
Get a direct answer with sources	Perplexity	"events in Berlin this weekend"
Know the current status of something	Perplexity	"what is the status of Ethereum upgrades"
Get real-time info	You.com	"latest AI regulation news"
Search without being tracked	SearXNG	anything, privately

Pro tip: Just search normally! Auto-routing handles most queries correctly. Override with -p provider when needed.

---

🧠 How Auto-Routing Works

The skill looks at your query and picks the best provider:

"iPhone 16 price"              → Serper (shopping keywords)
"how does quantum computing work" → Tavily (research question)
"companies like stripe.com"    → Exa (URL detected, similarity)
"events in Graz this weekend"  → Perplexity (local + direct answer)
"latest news on AI"            → You.com (real-time intent)
"search privately"             → SearXNG (privacy keywords)

What if it picks wrong? Override it: python3 scripts/search.py -p tavily -q "your query"

Debug routing: python3 scripts/search.py --explain-routing -q "your query"

---

📖 Usage Examples

Let Auto-Routing Choose (Recommended)

python3 scripts/search.py -q "Tesla Model 3 price"
python3 scripts/search.py -q "explain machine learning"
python3 scripts/search.py -q "startups like Figma"

Force a Specific Provider

python3 scripts/search.py -p serper -q "weather Berlin"
python3 scripts/search.py -p tavily -q "quantum computing" --depth advanced
python3 scripts/search.py -p exa --similar-url "https://stripe.com" --category company
python3 scripts/search.py -p you -q "breaking tech news" --include-news
python3 scripts/search.py -p searxng -q "linux distros" --engines "google,bing"

---

⚙ Configuration

{
  "auto_routing": {
    "enabled": true,
    "fallback_provider": "serper",
    "confidence_threshold": 0.3,
    "disabled_providers": []
  },
  "serper": {"country": "us", "language": "en"},
  "tavily": {"depth": "advanced"},
  "exa": {"type": "neural"},
  "you": {"country": "US", "include_news": true},
  "searxng": {"instance_url": "https://your-instance.example.com"}
}

---

📊 Provider Comparison

Feature	Serper	Tavily	Exa	Perplexity	You.com	SearXNG
Speed	⚡⚡⚡	⚡⚡	⚡⚡	⚡⚡	⚡⚡⚡	⚡⚡
Direct Answers	✗	✗	✗	✓✓	✗	✗
Citations	✗	✗	✗	✓	✗	✗
Factual Accuracy	⭐⭐⭐	⭐⭐⭐	⭐⭐	⭐⭐⭐	⭐⭐⭐	⭐⭐⭐
Semantic Understanding	⭐	⭐⭐	⭐⭐⭐	⭐⭐⭐	⭐⭐	⭐
Full Page Content	✗	✓	✓	✓	✓	✗
Shopping/Local	✓	✗	✗	✗	✗	✓
Find Similar Pages	✗	✗	✓	✗	✗	✗
RAG-Optimized	✗	✓	✗	✗	✓✓	✗
Privacy-First	✗	✗	✗	✗	✗	✓✓
API Cost	$$	$$	$$	Via Kilo	$	FREE

---

❓ Common Questions

Do I need API keys for all providers?

No. You only need keys for providers you want to use. Start with one (Serper recommended), add more later.

Which provider should I start with?

Serper — fastest, cheapest, largest free tier (2,500 queries/month), and handles most queries well.

What if I run out of free queries?

The skill automatically falls back to your other configured providers. Or switch to SearXNG (unlimited, self-hosted).

How much does this cost?

• Free tiers: 2,500 (Serper) + 1,000 (Tavily) + 1,000 (Exa) = 4,500+ free searches/month
• SearXNG: Completely free (just ~$5/mo if you self-host on a VPS)
• Paid plans: Start around $10-50/month depending on provider

Is SearXNG really private?

Yes, if self-hosted. You control the server, no tracking, no profiling. Public instances depend on the operator's policy.

How do I set up SearXNG?

# Docker (5 minutes)
docker run -d -p 8080:8080 searxng/searxng

Then enable JSON API in settings.yml. See docs.searxng.org.

Why did it route my query to the "wrong" provider?

Sometimes queries are ambiguous. Use --explain-routing to see why, then override with -p provider if needed.

---

🔄 Automatic Fallback

If one provider fails (rate limit, timeout, error), the skill automatically tries the next provider. You'll see routing.fallback_used: true in the response when this happens.

---

📤 Output Format

{
  "provider": "serper",
  "query": "iPhone 16 price",
  "results": [{"title": "...", "url": "...", "snippet": "...", "score": 0.95}],
  "routing": {
    "auto_routed": true,
    "provider": "serper",
    "confidence": 0.78,
    "confidence_level": "high"
  }
}

---

⚠ Important Note

Tavily, Serper, and Exa are NOT core OpenClaw providers.

❌ Don't modify ~/.openclaw/openclaw.json for these
✅ Use this skill's scripts — keys auto-load from .env

---

🔒 Security

SearXNG SSRF Protection: The SearXNG instance URL is validated with defense-in-depth:

• Enforces http/https schemes only
• Blocks cloud metadata endpoints (169.254.169.254, metadata.google.internal)
• Resolves hostnames and blocks private/internal IPs (loopback, RFC1918, link-local, reserved)
• Operators who intentionally self-host on private networks can set SEARXNG_ALLOW_PRIVATE=1

📚 More Documentation

• FAQ.md — Detailed answers to more questions
• TROUBLESHOOTING.md — Fix common errors
• README.md — Full technical reference

---

🔗 Quick Links

• Serper — Google Search API
• Tavily — AI Research Search
• Exa — Neural Search
• Perplexity — AI-Synthesized Answers (via Kilo Gateway)
• You.com — RAG/Real-time Search
• SearXNG — Privacy-First Meta-Search