Get your dream built 10x faster

How to integrate baidu web search with OpenClaw

We build custom applications 5x faster and cheaper 🚀

Book a Free Consultation
4.9
Clutch rating 🌟
600+
Happy partners
17+
Countries served
190+
Team members
Matt Graham, CEO of Rapid Developers

Book a call with an Expert

Stuck on an error? Book a 30-minute call with an engineer and get a direct fix + next steps. No pressure, no commitment.

How to integrate baidu web search with OpenClaw

The direct answer: build a small, authenticated, external search adapter (a proxy/microservice) that holds your Baidu credentials and provides a stable, authenticated REST API that an OpenClaw skill (configured in ClawHub) can call. Keep credentials and scopes in ClawHub secrets/environment variables, call the adapter from the skill runtime, and move caching, rate‑limit handling, pagination, and persistent indexing outside the agent runtime. Authenticate and validate every request, log and inspect API responses when things fail, and treat the agent as the thin orchestrator that invokes your external service — not the place for long-lived state or secret storage.

 

Overview: what this integration looks like

 
  • External adapter (recommended): a small web service you own that talks to Baidu’s web search API using credentials you store securely. It handles retries, rate limits, response normalization, and caching.
  • OpenClaw skill: a lightweight handler configured through ClawHub that calls your adapter’s REST endpoints. The skill passes user queries and returns normalized results to the agent runtime.
  • Secrets & config: store Baidu credentials (API key, client secret, OAuth tokens if used) and your adapter URL in ClawHub environment variables or secret storage; do not bake them into code.
  • Runtime separation: keep long-running tasks, persistence, and retry logic in external services. The skill makes short authenticated calls to your adapter and returns results.

 

Step-by-step: practical tasks

 
  • Register with Baidu: sign up in Baidu’s developer console and obtain the API credentials required for their web search product (API key, app id, or OAuth credentials depending on the product). Note the exact authentication method and request/response formats from Baidu’s official docs.
  • Design an adapter: implement a small HTTPS service (for example, Node/Express) that:
    • reads Baidu credentials from environment variables
    • accepts sanitized query parameters from the OpenClaw skill
    • forwards requests to the Baidu API, applying required auth headers or query params
    • normalizes the Baidu response into a predictable JSON shape for your skill
    • implements caching, rate-limit handling (backoff/retry), and logging
  • Secure the adapter: protect the adapter endpoint using API keys, mutual TLS, or signed requests so only your OpenClaw skill can call it. Validate inbound requests and throttle when needed.
  • Configure the skill in ClawHub: create the skill entry and set environment variables/secrets (adapter URL, adapter auth token). The skill code should be minimal: receive user intent, call your adapter, return results. Do not store Baidu credentials in the skill itself.
  • Implement logging & monitoring: log request/response IDs, HTTP status codes, and Baidu error payloads in both adapter and skill. Capture metrics for rate limits, latency, and error rates.
  • Test thoroughly: unit test the adapter against Baidu sandbox or a test account, exercise error conditions (401/403/429/500), and test the full flow from the OpenClaw skill through ClawHub to the adapter and back.

 

Minimal adapter example (Node.js + Express)

 

Use this as a starting point. Replace BAIDU_API_URL and BAIDU_API_KEY with values from Baidu docs. Store them in your deployment environment; do not commit them.

<b>//</b> server.js - minimal adapter that forwards to Baidu and normalizes results
const express = require('express');
const axios = require('axios');

const app = express();
app.use(express.json());

const BAIDU_API_URL = process.env.BAIDU_API_URL; <b>//</b> e.g. https://api.baidu.com/xxx (replace with Baidu doc URL)
const BAIDU_API_KEY = process.env.BAIDU_API_KEY;
const ADAPTER_TOKEN = process.env.ADAPTER_TOKEN; <b>//</b> token the OpenClaw skill will present

if (!BAIDU_API_URL || !BAIDU_API_KEY || !ADAPTER_TOKEN) {
  console.error('Missing required env vars: BAIDU_API_URL, BAIDU_API_KEY, ADAPTER_TOKEN');
  process.exit(1);
}

app.get('/search', async (req, res) => {
  try {
    const authHeader = req.header('Authorization') || '';
    if (authHeader !== `Bearer ${ADAPTER_TOKEN}`) {
      return res.status(401).json({ error: 'unauthorized' });
    }

    const q = (req.query.q || '').trim();
    if (!q) return res.status(400).json({ error: 'missing query param q' });

    <b>//</b> Forward request to Baidu API. Adjust params/headers per Baidu docs.
    const resp = await axios.get(BAIDU_API_URL, {
      params: {
        q,
        // add other params required by Baidu: page, size, safe, etc.
        // key: BAIDU_API_KEY // some APIs accept key param instead of header
      },
      headers: {
        'Ocp-Apim-Subscription-Key': BAIDU_API_KEY <b>//</b> or correct header per Baidu docs
      },
      timeout: 5000
    });

    <b>//</b> Normalize response shape for the skill
    const baiduData = resp.data;
    const normalized = {
      query: q,
      raw: baiduData,
      results: (baiduData.items || baiduData.results || []).map(item => ({
        title: item.title || item.h1 || '',
        snippet: item.snippet || item.summary || '',
        url: item.url || item.link || ''
      }))
    };

    res.json(normalized);
  } catch (err) {
    console.error('Adapter error', err.response ? err.response.data : err.message);
    const status = err.response ? err.response.status : 500;
    res.status(status).json({ error: 'search_error', details: err.response ? err.response.data : err.message });
  }
});

const port = process.env.PORT || 8080;
app.listen(port, () => console.log(`Adapter listening on ${port}`));

 

Minimal OpenClaw skill example (generic REST call)

 

This is a skill runtime snippet that calls your adapter. Store ADAPTER_URL and ADAPTER_TOKEN as ClawHub environment variables/secrets.

<b>//</b> skill-handler.js - run inside the OpenClaw skill runtime
async function handleSearchRequest(userQuery) {
  const adapterUrl = process.env.ADAPTER_URL; <b>//</b> e.g. https://your-adapter.example.com/search
  const token = process.env.ADAPTER_TOKEN;

  if (!userQuery) {
    return { error: 'no query provided' };
  }

  const url = `${adapterUrl}?q=${encodeURIComponent(userQuery)}`;
  const resp = await fetch(url, {
    method: 'GET',
    headers: {
      'Authorization': `Bearer ${token}`,
      'Accept': 'application/json'
    }
  });

  if (!resp.ok) {
    const body = await resp.text();
    throw new Error(`Adapter returned ${resp.status}: ${body}`);
  }

  const data = await resp.json();
  <b>//</b> Return normalized results to the agent or user
  return {
    query: data.query,
    results: data.results
  };
}

 

Security, authentication, and secrets

 
  • Keep Baidu credentials in the adapter only. The OpenClaw skill should never contain raw Baidu API keys unless those keys are stored as secure env vars in ClawHub and are meant for that runtime.
  • Use short-lived tokens if Baidu supports OAuth. If Baidu uses OAuth, implement token refresh in the adapter (server side) rather than in the skill runtime.
  • Authenticate calls from the skill to your adapter. Use an adapter-specific token stored in ClawHub secrets; verify it on the adapter. Consider mTLS for production.
  • Validate webhooks if Baidu sends asynchronous results — verify signatures from Baidu and verify incoming source IPs if documented.

 

Operational considerations

 
  • Caching: cache popular queries at the adapter layer to reduce cost and latency. Respect legal and freshness constraints in Baidu’s terms.
  • Rate limit handling: implement exponential backoff and circuit breakers in the adapter. Surface clear error messages to the skill when Baidu returns 429.
  • Pagination: map Baidu pagination to your normalized interface and expose safe ways for the skill to request more results.
  • Content filtering: apply safe-search/NSFW filtering either via Baidu parameters or at the adapter layer if your product requires it.
  • Monitoring: export metrics (requests, errors, latency) from the adapter; track Baidu quota usage.
  • Persistence: store indexes, logs, or user-facing histories outside the agent runtime (databases, object storage). Agents are ephemeral and not suitable for reliable persistence.

 

Debugging checklist

 
  • Confirm Baidu credentials are correct and have required scopes/quotas (check Baidu dev console).
  • Hit the adapter endpoint directly (curl/postman) with the adapter token to see raw Baidu responses and error payloads.
  • Inspect adapter logs for HTTP status codes, headers returned by Baidu (rate-limit headers), and response bodies.
  • Verify the skill’s environment variables in ClawHub match the adapter URL and token, and that the skill actually sends the Authorization header.
  • Reproduce errors locally: run the adapter and skill locally with the same env vars and use a known query to verify end-to-end.

Book Your Free 30‑Minute Migration Call

Speak one‑on‑one with a senior engineer about your no‑code app, migration goals, and budget. In just half an hour you’ll leave with clear, actionable next steps—no strings attached.

Book a Free Consultation

Troubleshooting baidu web search and OpenClaw Integration

1

Decode GBK/GB2312 for Baidu pages

Fetch the page as raw bytes and decode with a GBK/GB2312 codec (e.g. iconv‑lite in Node or .decode('gbk') in Python). Don’t rely only on headers—inspect the HTML meta charset or run a charset detector if needed.

  • Fetch raw bytes (fetch/axios with arrayBuffer/response.content).
  • Decode with iconv.decode(buf,'gbk') or Python requests.content.decode('gbk').
  • Verify via Content-Type or <meta charset>; fall back to chardet/jschardet.
// npm install iconv-lite
import iconv from 'iconv-lite';

async function fetchBaidu(url) {
  const res = await fetch(url, { headers: { 'User-Agent': 'Mozilla/5.0' } });
  const ab = await res.arrayBuffer();
  const buf = Buffer.from(ab);
  // decode as GBK (gb2312 maps to gbk)
  return iconv.decode(buf, 'gbk');
}

2

Avoid 302/CAPTCHA — UA rotation, proxies, throttling

Yes — build skills in ClawHub, run agents in the OpenClaw runtime, and connect external services by calling real APIs (REST/GraphQL/SDKs) with credentials stored in environment variables or OAuth tokens; validate webhooks, inspect logs and skill traces, and move stateful or scaling pieces (databases, queues) outside the agent runtime for production.

 

Practical steps

 
  • Provide credentials via env vars or OAuth and scope permissions.
  • Implement API clients inside a skill or external microservice; validate webhooks.
  • Debug with logs, API responses and skill execution traces; externalize stateful components.
const token = process.env.API_TOKEN; // read token
await fetch('https://api.example.com/data',{ // call API
  headers:{ Authorization:`Bearer ${token}` }
});

3

Enable headless JS rendering / update selectors

To enable headless JS rendering, run a headless browser (Playwright/Puppeteer) as an external service and point your OpenClaw skill to it via environment variables or skill configuration in ClawHub; update selectors to be resilient (data-* attributes, ARIA, stable classes) and add waits for network/DOM stability before scraping.

 

Enable headless JS rendering

 

Run a standalone headless renderer outside the agent runtime (container or service). Expose an HTTP/JSON endpoint (Playwright/Puppeteer server). Configure the skill in ClawHub or via env vars with that endpoint and auth token. Ensure skills request rendering, handle timeouts, and validate webhook/response integrity.

 

Update selectors

 

  • Prefer data-* or ARIA attributes
  • Use explicit waits for network/DOM
  • Fallback strategies: CSS → XPath → text match

4

Configure Baidu API key, signature, request params

Set your Baidu credentials as environment variables (for example BAIDU_API_KEY/BAIDU_SECRET_KEY or BAIDU_APPID/BAIDU_SECRET_KEY). Obtain an access_token from Baidu’s OAuth endpoint for services that require it, and for Baidu Translate compute a sign = MD5(appid + q + salt + secret_key). Include either access_token (query/body) or sign/salt in the request, use UTF-8, and protect keys in your runtime (env vars or secret store).

 

Details and example

 

Steps:

  • Export env vars and never hard-code keys.
  • Get token from /oauth/2.0/token with client_credentials.
  • Compute sign for Translate: MD5(appid+q+salt+secret_key).
```js // Node.js example const crypto = require('crypto'); const appid = process.env.BAIDU_APPID, key = process.env.BAIDU_SECRET_KEY; const q = 'hello', salt = Date.now(); const sign = crypto.createHash('md5').update(appid+q+salt+key).digest('hex'); // // call translate with params appid, q, salt, sign ```
Book a Free Consultation

Still stuck?
Copy this prompt into ChatGPT and get a clear, personalized explanation.

This prompt helps an AI assistant understand your setup and guide you through the fix step by step, without assuming technical knowledge.

AI AI Prompt


Recognized by the best

Trusted by 600+ businesses globally

From startups to enterprises and everything in between, see for yourself our incredible impact.

RapidDev was an exceptional project management organization and the best development collaborators I've had the pleasure of working with.

They do complex work on extremely fast timelines and effectively manage the testing and pre-launch process to deliver the best possible product. I'm extremely impressed with their execution ability.

Arkady
CPO, Praction
Working with Matt was comparable to having another co-founder on the team, but without the commitment or cost.

He has a strategic mindset and willing to change the scope of the project in real time based on the needs of the client. A true strategic thought partner!

Donald Muir
Co-Founder, Arc
RapidDev are 10/10, excellent communicators - the best I've ever encountered in the tech dev space.

They always go the extra mile, they genuinely care, they respond quickly, they're flexible, adaptable and their enthusiasm is amazing.

Mat Westergreen-Thorne
Co-CEO, Grantify
RapidDev is an excellent developer for custom-code solutions.

We’ve had great success since launching the platform in November 2023. In a few months, we’ve gained over 1,000 new active users. We’ve also secured several dozen bookings on the platform and seen about 70% new user month-over-month growth since the launch.

Emmanuel Brown
Co-Founder, Church Real Estate Marketplace
Matt’s dedication to executing our vision and his commitment to the project deadline were impressive. 

This was such a specific project, and Matt really delivered. We worked with a really fast turnaround, and he always delivered. The site was a perfect prop for us!

Samantha Fekete
Production Manager, Media Production Company
The pSEO strategy executed by RapidDev is clearly driving meaningful results.

Working with RapidDev has delivered measurable, year-over-year growth. Comparing the same period, clicks increased by 129%, impressions grew by 196%, and average position improved by 14.6%. Most importantly, qualified contact form submissions rose 350%, excluding spam.

Appreciation as well to Matt Graham for championing the collaboration!

Michael W. Hammond
Principal Owner, OCD Tech

We put the rapid in RapidDev

Need a dedicated strategic tech and growth partner? Discover what RapidDev can do for your business! Book a call with our team to schedule a free, no-obligation consultation. We’ll discuss your project and provide a custom quote at no cost.Â