Competitor price monitoring without misleading comparisons
A price chart is only useful when it compares the same product under the same conditions. Build a monitoring pipeline that makes matching, availability and missing observations explicit.
Define a comparable product set
Start with a manageable group of products that matter to your business. Record your own SKU and the competitor’s stable identifier, URL, brand, size and pack quantity. Manufacturer identifiers can help, but inspect their coverage and consistency before assuming they are unique.
Do not join on titles alone. “Travel bottle” can describe several sizes, bundles and materials. Match variants and quantities explicitly, and send uncertain matches for review. Keep the matching decision and evidence so future reports do not quietly switch products.
Collect the conditions around the price
Choose an abotapi scraper with the fields your comparison requires. Inspect its current output and test a small scope. You may need details for variants, availability, seller or delivery information. If the source does not expose a field, mark it unknown rather than filling in a convenient assumption.
Keep the displayed currency, source location, collection time and availability next to every price. Distinguish regular and sale prices when the source provides both. Membership offers, subscriptions, shipping and taxes can make two displayed numbers incomparable; describe what your metric includes and excludes.
Calculate changes on matched snapshots
Build a normalized table with product_id, observed_at, price and currency. Parse numeric values carefully and reject unparseable prices. Group currency values separately unless you explicitly apply an exchange-rate source and preserve the rate date. Never average unrelated currencies.
The example below assumes you already created matched, one-row-per-product snapshots in the same currency. The outer join keeps gaps visible. The percentage calculation applies only when both positive prices exist; absence is not converted to zero.
import pandas as pd
previous = pd.read_csv("previous.csv")
current = pd.read_csv("current.csv")
key = ["product_id", "currency"]
comparison = previous.merge(current, on=key, how="outer",
suffixes=("_before", "_after"), validate="one_to_one")
for name in ["price_before", "price_after"]:
comparison[name] = pd.to_numeric(comparison[name], errors="coerce")
valid = comparison.price_before.gt(0) & comparison.price_after.gt(0)
comparison.loc[valid, "change_pct"] = (
comparison.loc[valid, "price_after"] / comparison.loc[valid, "price_before"] - 1
) * 100
comparison.to_csv("price_changes.csv", index=False)Design alerts around meaningful evidence
Agree on which changes deserve attention: a sustained discount on a matched item, a competitor returning to stock, or a price gap above a business-defined threshold. Use your margin and decision cadence to set thresholds. There is no single alert percentage that works for every category.
Recheck surprising observations before taking action. A large movement can be a pack-size mismatch, an incomplete page, a regional offer or a collection error. Include before and after values, currency, timestamps and source links in the alert. If collection failed, send a coverage alert instead of a price-change alert.
Schedule collection at a cadence that fits the decision. More frequent runs add observations and cost without necessarily adding value. Inspect the actor’s current price and set run limits before expanding the scope.
import csv
import math
import sys
from pathlib import Path
threshold = float(sys.argv[1])
if not math.isfinite(threshold) or threshold <= 0:
raise ValueError("Threshold must be a positive percentage")
with Path("price_changes.csv").open(newline="") as source:
for row in csv.DictReader(source):
try:
change = float(row["change_pct"])
except (ValueError, KeyError, TypeError):
continue
if math.isfinite(change) and abs(change) >= threshold:
print(row["product_id"], row["currency"], f"{change:+.1f}%")Report uncertainty alongside the opportunity
Show the number of products successfully matched and observed in both snapshots. Separate missing observations, unavailable products and confirmed price changes. Segment by brand and category only when the groups have enough comparable observations to support the conclusion.
A monitoring report helps your team investigate changes; it does not prove why a competitor changed its price. Preserve the raw snapshots and matching rules, then combine the findings with your commercial context before changing a campaign or repricing products.
Put the guide into practice
Choose a scraper for your source, inspect a small dataset, and agree on the checks before scaling.