# CrawlConsole -- Bolt.new Install Guide

This guide installs CrawlConsole tracking into a Bolt.new website using Bolt's
built-in server-side functions. It does **not** require Cloudflare Workers,
Cloudflare proxying, or any external reverse proxy.

Bolt sites are commonly **Vite + React** apps. The public page runs in the
browser, while sensitive server work belongs in a Bolt Database Server Function
(also called an edge function). For CrawlConsole, the Bolt-native setup is:

```txt
Bolt React app
  -> calls a Bolt Database Server Function
  -> Server Function reads CrawlConsole secrets
  -> Server Function sends the tracking event to CrawlConsole
```

This keeps `CRAWLCONSOLE_TRACKER_KEY` out of the browser. The browser only calls
your own Bolt server function.

Important limitation: a Vite/React page hosted on Bolt does not expose an
Express-style middleware file where every raw page request automatically passes
through your code. This guide is the Bolt-native implementation: the tracking
logic runs server-side in a Bolt Server Function, and the app triggers that
function on page load/navigation.

---

## 1. Add the two secrets in Bolt

Open your Bolt project, then open the database settings:

1. Click the database icon in the top center of the Bolt project.
2. Open **Secrets**.
3. Add these two secrets:

| Name                          | Value                         |
| ----------------------------- | ----------------------------- |
| `CRAWLCONSOLE_PROJECT_KEY`    | your CrawlConsole project key |
| `CRAWLCONSOLE_TRACKER_KEY`    | your CrawlConsole tracker key |

Rules:

- **Do NOT prefix with `VITE_`**. These are runtime secrets for the server
  function, not browser environment variables.
- Names must match exactly. The server function reads
  `Deno.env.get("CRAWLCONSOLE_PROJECT_KEY")` and
  `Deno.env.get("CRAWLCONSOLE_TRACKER_KEY")`.
- Do not put `CRAWLCONSOLE_TRACKER_KEY` in client code.

---

## 2. Create the `crawlconsole-track` Server Function

Create `supabase/functions/crawlconsole-track/index.ts`:

```ts
import "jsr:@supabase/functions-js/edge-runtime.d.ts";

const corsHeaders = {
  "Access-Control-Allow-Origin": "*",
  "Access-Control-Allow-Methods": "POST, OPTIONS",
  "Access-Control-Allow-Headers": "Content-Type, Authorization, X-Client-Info, Apikey",
};

const CRAWLCONSOLE_ENDPOINT = "https://analytics.crawlconsole.com/v1/track";

const SKIP_EXTENSIONS = [
  ".css",
  ".js",
  ".map",
  ".png",
  ".jpg",
  ".jpeg",
  ".svg",
  ".webp",
  ".ico",
  ".woff",
  ".woff2",
  ".ttf",
];

function shouldSkip(pathname: string): boolean {
  if (
    pathname.startsWith("/assets/") ||
    pathname.startsWith("/_build/") ||
    pathname === "/favicon.ico" ||
    pathname === "/robots.txt"
  ) {
    return true;
  }

  return SKIP_EXTENSIONS.some((ext) => pathname.endsWith(ext));
}

function clientIp(req: Request, clientIpRaw: string): string {
  return (
    req.headers.get("cf-connecting-ip") ||
    req.headers.get("x-real-ip") ||
    (req.headers.get("x-forwarded-for") || "").split(",")[0].trim() ||
    clientIpRaw.split(",")[0].trim() ||
    ""
  );
}

Deno.serve(async (req: Request) => {
  if (req.method === "OPTIONS") {
    return new Response(null, { status: 200, headers: corsHeaders });
  }

  if (req.method !== "POST") {
    return new Response(JSON.stringify({ error: "Method not allowed" }), {
      status: 405,
      headers: { ...corsHeaders, "Content-Type": "application/json" },
    });
  }

  try {
    const projectKey = Deno.env.get("CRAWLCONSOLE_PROJECT_KEY") || "";
    const trackerKey = Deno.env.get("CRAWLCONSOLE_TRACKER_KEY") || "";

    if (!projectKey || !trackerKey) {
      console.warn(
        "CrawlConsole tracking is missing CRAWLCONSOLE_PROJECT_KEY or CRAWLCONSOLE_TRACKER_KEY.",
      );

      return new Response(JSON.stringify({ ok: false, missing_config: true }), {
        status: 200,
        headers: { ...corsHeaders, "Content-Type": "application/json" },
      });
    }

    const body = await req.json().catch(() => ({}));
    const {
      path = "/",
      full_path = "",
      user_agent = "",
      status_code = 200,
      client_ip_raw = "",
    } = body;

    let pathname: string;
    try {
      pathname = new URL(String(path), "https://example.com").pathname;
    } catch {
      pathname = "/";
    }

    if (shouldSkip(pathname)) {
      return new Response(JSON.stringify({ skipped: true }), {
        status: 200,
        headers: { ...corsHeaders, "Content-Type": "application/json" },
      });
    }

    await fetch(CRAWLCONSOLE_ENDPOINT, {
      method: "POST",
      headers: {
        authorization: `Bearer ${trackerKey}`,
        "content-type": "application/json",
      },
      body: JSON.stringify({
        project_key: projectKey,
        user_agent: String(user_agent || req.headers.get("user-agent") || ""),
        path: String(path),
        full_path: String(full_path),
        status_code: Number(status_code) || 200,
        client_ip: clientIp(req, String(client_ip_raw)),
        client_ip_raw: String(client_ip_raw || req.headers.get("x-forwarded-for") || ""),
      }),
    });

    return new Response(JSON.stringify({ ok: true }), {
      status: 200,
      headers: { ...corsHeaders, "Content-Type": "application/json" },
    });
  } catch (err) {
    console.warn("CrawlConsole tracking failed", err);

    return new Response(JSON.stringify({ ok: false }), {
      status: 200,
      headers: { ...corsHeaders, "Content-Type": "application/json" },
    });
  }
});
```

Deploy it from Bolt as a Server Function named:

```txt
crawlconsole-track
```

If Bolt asks whether the function requires authentication, make it public or set
JWT verification to false. The browser needs to be able to call this function
without a signed-in user session.

---

## 3. Add the client tracking helper

Create `src/lib/crawlconsole-tracking.ts`:

```ts
// src/lib/crawlconsole-tracking.ts

const SUPABASE_URL = import.meta.env.VITE_SUPABASE_URL;
const SUPABASE_ANON_KEY = import.meta.env.VITE_SUPABASE_ANON_KEY;

const FUNCTION_URL = `${SUPABASE_URL}/functions/v1/crawlconsole-track`;

const SKIP_EXTENSIONS = [
  ".css",
  ".js",
  ".map",
  ".png",
  ".jpg",
  ".jpeg",
  ".svg",
  ".webp",
  ".ico",
  ".woff",
  ".woff2",
  ".ttf",
];

function shouldSkip(pathname: string): boolean {
  if (
    pathname.startsWith("/assets/") ||
    pathname.startsWith("/_build/") ||
    pathname === "/favicon.ico" ||
    pathname === "/robots.txt"
  ) {
    return true;
  }

  return SKIP_EXTENSIONS.some((ext) => pathname.endsWith(ext));
}

function send(payload: Record<string, unknown>): void {
  const body = JSON.stringify(payload);

  if (navigator.sendBeacon) {
    const blob = new Blob([body], { type: "application/json" });
    navigator.sendBeacon(FUNCTION_URL, blob);
    return;
  }

  fetch(FUNCTION_URL, {
    method: "POST",
    headers: {
      Authorization: `Bearer ${SUPABASE_ANON_KEY}`,
      "Content-Type": "application/json",
    },
    body,
    keepalive: true,
  }).catch(() => undefined);
}

export function trackPageView(statusCode = 200): void {
  if (!SUPABASE_URL || !SUPABASE_ANON_KEY) return;

  const url = new URL(window.location.href);
  if (shouldSkip(url.pathname)) return;

  const path = `${url.pathname}${url.search}`;

  send({
    path,
    full_path: `${url.origin}${path}`,
    user_agent: navigator.userAgent,
    status_code: statusCode,
  });
}
```

Note: `VITE_SUPABASE_URL` and `VITE_SUPABASE_ANON_KEY` are public client
configuration values used to call your own Bolt/Supabase function. The private
CrawlConsole tracker key remains only in Bolt Secrets.

---

## 4. Call `trackPageView()` on page load and navigation

In `src/App.tsx`, call the helper when the page loads:

```tsx
import { useEffect } from "react";
import { trackPageView } from "./lib/crawlconsole-tracking";

function App() {
  useEffect(() => {
    trackPageView();
  }, []);

  return (
    <main>
      {/* your app */}
    </main>
  );
}

export default App;
```

If your Bolt app uses React Router, call it again when the route changes:

```tsx
import { useEffect } from "react";
import { useLocation } from "react-router-dom";
import { trackPageView } from "./lib/crawlconsole-tracking";

function CrawlConsoleTracker() {
  const location = useLocation();

  useEffect(() => {
    trackPageView();
  }, [location.pathname, location.search]);

  return null;
}
```

Render `<CrawlConsoleTracker />` inside your router.

---

## 5. Verify the install

Publish the Bolt app, then open the site in a browser:

```txt
https://your-site.bolt.host
```

Open Bolt's Server Function logs for `crawlconsole-track` and confirm requests
are arriving. Then check CrawlConsole for page requests.

To test the Server Function directly, send a sample event to the function URL:

```bash
curl -X POST "$VITE_SUPABASE_URL/functions/v1/crawlconsole-track" \
  -H "Authorization: Bearer $VITE_SUPABASE_ANON_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "path": "/",
    "full_path": "https://your-site.bolt.host/",
    "user_agent": "Googlebot/2.1 (+http://www.google.com/bot.html)",
    "status_code": 200
  }'
```

Do not use `curl https://your-site.bolt.host/` as the only verification for a
Vite/React site. Curl does not execute browser JavaScript, so it will not trigger
`trackPageView()`. The CrawlConsole event is sent server-side from the Bolt
Server Function after the browser app calls it, and the tracker key stays
private.

---

## Summary

| Step | Bolt-native setup |
| ---- | ----------------- |
| 1 | Add `CRAWLCONSOLE_PROJECT_KEY` and `CRAWLCONSOLE_TRACKER_KEY` in Bolt Secrets |
| 2 | Create the public `crawlconsole-track` Bolt Server Function |
| 3 | Add `src/lib/crawlconsole-tracking.ts` in the React app |
| 4 | Call `trackPageView()` on load and route changes |

The key difference from Lovable: Lovable can run tracking in a Cloudflare Worker
request handler. Bolt's built-in path is a Server Function. The tracking event
is sent from server-side Bolt code, while the Vite/React app triggers that server
function.