Jak zbudować bota do monitorowania cen konkurencji

Inicjalizujemy projekt TypeScript z Playwright i kilkoma helperami:

mkdir price-monitor && cd price-monitor
npm init -y
npm i playwright pg zod pino
npm i -D typescript @types/node tsx
npx playwright install chromium

Czemu te zależności:

• playwright — browser automation z auto-wait
• pg — PostgreSQL client (lepiej niż SQLite jeśli planujesz scale)
• zod — runtime schema validation (wykryje gdy parser się rozjedzie)
• pino — strukturalne logging (JSON-line dla observability)

Każdy target (sklep) ma własną strukturę — definiujemy to deklaratywnie:

// config/targets.ts
export const targets = [
  {
    id: 'allegro',
    baseUrl: 'https://allegro.pl/oferta/',
    selectors: {
      price: '[data-role="app-container"] [aria-label*="cena"]',
      title: 'h1',
      availability: '[data-role="availability"]',
    },
    waitFor: 'h1',
    proxyTier: 'residential',
  },
  // ...
];

Schema dla wyniku (Zod):

import { z } from 'zod';
export const PriceData = z.object({
  sku: z.string(),
  targetId: z.string(),
  price: z.number().positive(),
  currency: z.enum(['PLN', 'EUR', 'USD']),
  available: z.boolean(),
  scrapedAt: z.date(),
});

Schema validation jest krytyczna — gdy sklep zmieni layout i selector zwróci śmieci, validation zawiedzie zamiast wepchnąć złe dane do bazy.

Playwright używa browser contexts — to jak osobne incognito sessions z izolowanymi cookies/storage. Dla każdego SKU tworzymy nowy context z innym proxy:

// scraper/browser.ts
import { chromium } from 'playwright';

const proxies = [
  'http://user:pass@proxy1.smartproxy.com:7000',
  'http://user:pass@proxy2.smartproxy.com:7000',
  // ...
];

export async function scrapeWithRotation(url, selectors) {
  const proxy = proxies[Math.floor(Math.random() * proxies.length)];
  const browser = await chromium.launch({
    headless: true,
    proxy: { server: proxy },
  });
  const context = await browser.newContext({
    userAgent: getRandomUserAgent(),
    viewport: { width: 1920, height: 1080 },
    locale: 'pl-PL',
  });
  // ... scrape logic
  await browser.close();
}

Dla protected sites (Allegro, Amazon) dodaj playwright-extra + puppeteer-extra-plugin-stealth żeby ukryć headless flagi.

Każdy scrape może paść: timeout, captcha, network error, parser drift. Implementacja exponential backoff:

async function scrapeWithRetry(target, sku, maxAttempts = 5) {
  let lastError;
  for (let attempt = 1; attempt <= maxAttempts; attempt++) {
    try {
      const data = await scrape(target, sku);
      return PriceData.parse(data); // validate
    } catch (err) {
      lastError = err;
      const delay = Math.min(2 ** attempt * 1000, 30000);
      logger.warn({ sku, attempt, err: err.message });
      await sleep(delay);
    }
  }
  // After 5 fails → dead-letter queue
  await db.query(
    'INSERT INTO scrape_dlq (sku, target, error, failed_at) VALUES ($1, $2, $3, NOW())',
    [sku, target.id, lastError.message]
  );
  throw lastError;
}

DLQ z alertem na Slack gdy lista urośnie powyżej threshold — wiesz że coś się popsuło zanim klient zapyta "gdzie moje raporty".

Każdy scrape to nowy wiersz w tabeli historii. Wykrywanie zmiany cena = comparison vs previous:

CREATE TABLE price_history (
  id BIGSERIAL PRIMARY KEY,
  sku TEXT NOT NULL,
  target_id TEXT NOT NULL,
  price NUMERIC(10,2) NOT NULL,
  currency TEXT NOT NULL,
  available BOOLEAN NOT NULL,
  scraped_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE INDEX idx_price_lookup ON price_history (sku, target_id, scraped_at DESC);

Query do wykrywania spadku:

SELECT current.sku, current.price, prev.price as prev_price,
       ((current.price - prev.price) / prev.price * 100) as pct_change
FROM price_history current
JOIN LATERAL (
  SELECT price FROM price_history
  WHERE sku = current.sku AND target_id = current.target_id
    AND scraped_at < current.scraped_at
  ORDER BY scraped_at DESC LIMIT 1
) prev ON true
WHERE current.scraped_at > NOW() - INTERVAL '1 hour'
  AND ((current.price - prev.price) / prev.price) < -0.05;

Każdy wiersz w wyniku = spadek >5%. Alert do Slacka.

Webhook na Slack (incoming webhook URL z Slack App):

async function sendSlackAlert(changes) {
  await fetch(process.env.SLACK_WEBHOOK_URL, {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({
      text: `Price drops detected: ${changes.length} SKUs`,
      blocks: changes.map(c => ({
        type: 'section',
        text: { type: 'mrkdwn',
          text: `• ${c.sku} (${c.target_id}): ${c.prev_price} → ${c.price} (${c.pct_change.toFixed(1)}%)`
        }
      }))
    })
  });
}

Scheduling: cron w VPS lub BullMQ + Redis dla dystrybuowanej kolejki. Co 30 min:

// cron: */30 * * * *
import { scrapeAll } from './scraper';
import { detectChanges } from './detect';
import { sendSlackAlert } from './alerts';

(async () => {
  await scrapeAll(targets, getSkuList());
  const changes = await detectChanges();
  if (changes.length > 0) await sendSlackAlert(changes);
})();