How to build a competitor price monitoring bot

Initialize a TypeScript project with Playwright and a few helpers:

mkdir price-monitor && cd price-monitor
npm init -y
npm i playwright pg zod pino
npm i -D typescript @types/node tsx
npx playwright install chromium

Why these dependencies:

• playwright — browser automation with auto-wait
• pg — PostgreSQL client (better than SQLite if you plan to scale)
• zod — runtime schema validation (catches parser drift)
• pino — structured logging (JSON-line for observability)

Each target (store) has its own structure — we define it declaratively:

// config/targets.ts
export const targets = [
  {
    id: 'amazon-de',
    baseUrl: 'https://www.amazon.de/dp/',
    selectors: {
      price: '#corePrice_feature_div .a-offscreen',
      title: '#productTitle',
      availability: '#availability',
    },
    waitFor: '#productTitle',
    proxyTier: 'residential',
  },
  // ...
];

Schema for the result (Zod):

import { z } from 'zod';
export const PriceData = z.object({
  sku: z.string(),
  targetId: z.string(),
  price: z.number().positive(),
  currency: z.enum(['PLN', 'EUR', 'USD']),
  available: z.boolean(),
  scrapedAt: z.date(),
});

Schema validation is critical — when the store changes layout and a selector returns garbage, validation fails instead of pushing bad data into the database.

Playwright uses browser contexts — like separate incognito sessions with isolated cookies/storage. For each SKU we create a new context with a different proxy:

// scraper/browser.ts
import { chromium } from 'playwright';

const proxies = [
  'http://user:pass@proxy1.smartproxy.com:7000',
  'http://user:pass@proxy2.smartproxy.com:7000',
  // ...
];

export async function scrapeWithRotation(url, selectors) {
  const proxy = proxies[Math.floor(Math.random() * proxies.length)];
  const browser = await chromium.launch({
    headless: true,
    proxy: { server: proxy },
  });
  const context = await browser.newContext({
    userAgent: getRandomUserAgent(),
    viewport: { width: 1920, height: 1080 },
    locale: 'en-US',
  });
  // ... scrape logic
  await browser.close();
}

For protected sites (Amazon, eBay) add playwright-extra + puppeteer-extra-plugin-stealth to hide headless flags.

Every scrape can fail: timeout, captcha, network error, parser drift. Implement exponential backoff:

async function scrapeWithRetry(target, sku, maxAttempts = 5) {
  let lastError;
  for (let attempt = 1; attempt <= maxAttempts; attempt++) {
    try {
      const data = await scrape(target, sku);
      return PriceData.parse(data); // validate
    } catch (err) {
      lastError = err;
      const delay = Math.min(2 ** attempt * 1000, 30000);
      logger.warn({ sku, attempt, err: err.message });
      await sleep(delay);
    }
  }
  // After 5 fails → dead-letter queue
  await db.query(
    'INSERT INTO scrape_dlq (sku, target, error, failed_at) VALUES ($1, $2, $3, NOW())',
    [sku, target.id, lastError.message]
  );
  throw lastError;
}

DLQ with Slack alert when the list grows above threshold — you know something broke before the client asks "where are my reports".

Each scrape is a new row in the history table. Price change detection = comparison vs previous:

CREATE TABLE price_history (
  id BIGSERIAL PRIMARY KEY,
  sku TEXT NOT NULL,
  target_id TEXT NOT NULL,
  price NUMERIC(10,2) NOT NULL,
  currency TEXT NOT NULL,
  available BOOLEAN NOT NULL,
  scraped_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE INDEX idx_price_lookup ON price_history (sku, target_id, scraped_at DESC);

Query for drop detection:

SELECT current.sku, current.price, prev.price as prev_price,
       ((current.price - prev.price) / prev.price * 100) as pct_change
FROM price_history current
JOIN LATERAL (
  SELECT price FROM price_history
  WHERE sku = current.sku AND target_id = current.target_id
    AND scraped_at < current.scraped_at
  ORDER BY scraped_at DESC LIMIT 1
) prev ON true
WHERE current.scraped_at > NOW() - INTERVAL '1 hour'
  AND ((current.price - prev.price) / prev.price) < -0.05;

Each row in the result = drop >5%. Slack alert.

Slack webhook (incoming webhook URL from Slack App):

async function sendSlackAlert(changes) {
  await fetch(process.env.SLACK_WEBHOOK_URL, {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({
      text: `Price drops detected: ${changes.length} SKUs`,
      blocks: changes.map(c => ({
        type: 'section',
        text: { type: 'mrkdwn',
          text: `• ${c.sku} (${c.target_id}): ${c.prev_price} → ${c.price} (${c.pct_change.toFixed(1)}%)`
        }
      }))
    })
  });
}

Scheduling: cron on a VPS or BullMQ + Redis for distributed queue. Every 30 min:

// cron: */30 * * * *
import { scrapeAll } from './scraper';
import { detectChanges } from './detect';
import { sendSlackAlert } from './alerts';

(async () => {
  await scrapeAll(targets, getSkuList());
  const changes = await detectChanges();
  if (changes.length > 0) await sendSlackAlert(changes);
})();