SearXNG Search¶

SearXNG search provides functionality to perform web searches using the SearXNG meta search engine.

Class Overview¶

• SearXNGSearch - A class that provides SearXNG search functionality

Detailed API¶

SearXNGSearch Class¶

SearXNGSearch is a class that provides SearXNG search functionality, inheriting from BaseSearch.

Initialization Parameters¶

• base_url: Optional[str] = None - Base URL of the SearXNG instance, defaults to SEARXNG_URL environment variable
• api_key: Optional[str] = None - Optional API key for instances that protect the JSON API, defaults to SEARXNG_API_KEY environment variable

Methods¶

• search(query: str, number_results: int = 5, timeout: Optional[float] = None, **kwargs) -> List[SearchResult]: Execute search and return results
• _search_impl(query: str, **kwargs) -> List[SearchResult]: Implement specific search logic
• _parse_results(raw_results: Dict) -> List[SearchResult]: Parse raw search results

Free Tier¶

SearXNG offers unique advantages for free usage:

• Completely free and open source
• Self-hosted solution - no API limits
• Privacy-focused - no tracking or data collection
• Unlimited searches when self-hosted

Deployment Options¶

1. Private Instance: Deploy your own SearXNG instance for unlimited free usage
2. Public Instances: Use existing public instances (may have rate limits)
3. Residential IP Proxy: For agentic search, consider using residential IP proxy to prevent bot detection

Cost Considerations¶

• Self-hosting costs: Server hosting and maintenance
• Proxy costs: Optional residential IP proxy for enhanced reliability
• No API fees: No per-query charges unlike commercial providers

Environment Variables¶

When the SEARXNG_API_KEY is set, it is automatically included as an X-API-Key header in all requests. This is useful for self-hosted instances that use the SearXNG limiter to restrict JSON API access to authorized clients only.

Usage Examples¶

Basic Usage¶

from toolregistry_hub.websearch import SearXNGSearch

# Create SearXNG search instance
searxng_search = SearXNGSearch()

# Execute search
results = searxng_search.search("Python programming", number_results=5)

# Process search results
for result in results:
    print(f"Title: {result.title}")
    print(f"URL: {result.url}")
    print(f"Excerpt: {result.excerpt}")
    print("-" * 50)

Using Custom SearXNG Instance¶

from toolregistry_hub.websearch import SearXNGSearch

# Create search instance using custom SearXNG instance
searxng_search = SearXNGSearch(base_url="https://your-searxng-instance.com")

# Execute search
results = searxng_search.search("machine learning tutorial", number_results=3)

# Process search results
for result in results:
    print(f"Title: {result.title}")
    print(f"URL: {result.url}")
    print(f"Excerpt: {result.excerpt}")
    print("-" * 50)

Using API Key Authentication¶

from toolregistry_hub.websearch import SearXNGSearch

# Pass API key explicitly
searxng_search = SearXNGSearch(
    base_url="https://your-searxng-instance.com",
    api_key="your-api-key",
)

# Or set environment variables and use defaults:
#   export SEARXNG_URL="https://your-searxng-instance.com"
#   export SEARXNG_API_KEY="your-api-key"
# searxng_search = SearXNGSearch()

results = searxng_search.search("Python programming", max_results=5)

Setting Timeout¶

from toolregistry_hub.websearch import SearXNGSearch

# Create SearXNG search instance
searxng_search = SearXNGSearch()

# Execute search with timeout
results = searxng_search.search("deep learning frameworks", number_results=5, timeout=10.0)

# Process search results
for result in results:
    print(f"Title: {result.title}")
    print(f"URL: {result.url}")
    print(f"Excerpt: {result.excerpt}")
    print("-" * 50)

Fetching Web Page Content¶

from toolregistry_hub.websearch import SearXNGSearch
from toolregistry_hub.websearch.base import BaseSearch

# Create SearXNG search instance
searxng_search = SearXNGSearch()

# Execute search
results = searxng_search.search("Python tutorial", number_results=1)

if results:
    # Get full web page content of the first result
    url = results[0].url
    content = BaseSearch._fetch_webpage_content(url)
    print(f"Web page content length: {len(content)} characters")
    print(f"Web page content preview: {content[:200]}...")

Instance Configuration¶

The hub sends JSON API requests via HTTP POST. Your SearXNG instance must have json listed in its settings.yml under search.formats:

search:
  formats:
    - html
    - json    # required for the hub to work

Using POST avoids the Sec-Fetch-* header checks that SearXNG's built-in limiter enforces on GET requests, ensuring compatibility with instances that have limiter: true (the default since SearXNG 2023.x).

Introduction to SearXNG¶

SearXNG is a self-hosted meta search engine that can aggregate results from multiple search engines, providing a privacy-protected search experience. The main advantages of using SearXNG include:

1. Privacy Protection - SearXNG does not track users and does not store search history
2. Multi-engine Aggregation - Can get results from multiple search engines simultaneously
3. Self-hosted - Can deploy SearXNG instances on your own server
4. Customizable - Can customize search engines, result presentation, etc.