Shop It Docs
TradingView UDF

Data Flow

How NEPSE market data travels from the upstream MDP API to the TradingView chart

The TV module is purely read-side. This page shows where the data it serves comes from, what tables hold it, and how each piece gets refreshed.

End-to-end pipeline

Sources

The scheduler polls MDP during NPT trading hours and UPSERTs each tick into nepse_intraday_prices (companies), nepse_intraday_indices (top-level indices), and nepse_intraday_sub_indices (sector sub-indices).

Companies are keyed by (symbol, polled_at); indices and sub-indices by (index_id, polled_at) and (sub_index_id, polled_at) respectively.

TableKeyNotes
nepse_intraday_prices(symbol, polled_at)2000+ tickers
nepse_intraday_indices(index_id, polled_at)4 rows per poll — ^NEPSE, ^SENSITIVE, ^FLOAT, ^SFLOAT
nepse_intraday_sub_indices(sub_index_id, polled_at)13 rows per poll — sector indices

Rows older than the retention window (currently 3 trading days) are pruned by the cleanup cron (runs 12:45 AM NPT Mon–Fri).

When /history gets an intraday request, GetIntradayStockCandles / GetIntradayIndexCandles / GetIntradaySubIndexCandles aggregate these raw ticks into buckets of interval_secs seconds in a single SQL pass — no materialized views.

The EOD cron (3:15 PM NPT Mon–Fri, after session close at 15:00) snapshots each active company's final figures and INSERTs one row per symbol into nepse_price_history.

ColumnTypeNotes
company_iduuidFK to nepse_companies.id
trading_datedateEOD date
open_price, high_price, low_pricenumeric(12,2)
ltpnumeric(12,2)last traded price = EOD close
volumeintwhole shares
turnover, avg_price, year_high, year_low, market_capvarious

This table is permanent. Weekly and monthly /history aggregate directly from here; there's no separate weekly/monthly storage.

nepse_companies is the authoritative symbol list. Seeded once and reconciled on startup (company:bootstrap-sync task) and daily.

ColumnTypeNotes
iduuidprimary key
symboltextticker, unique
nametextcompany name
instrument_typetext
sector_iduuidFK to nepse_sectors
statuschar'A' = active, 'D' = delisted
image_url, isin_no, nepse_company_id, nepse_security_idvarious

Only status = 'A' rows are returned from /symbols and /search.

nepse_indices (4 rows) and nepse_sub_indices (13 rows) are master tables the poller upserts every tick. For TradingView they contribute a single column each: tv_symbol.

ColumnTypeNotes
iduuidprimary key
index_name / namevarchar(100)human label — "NEPSE Index", "Banking", ...
tv_symboltext NOT NULL UNIQUEshort ticker for TV, no ^. The service prepends ^ when serializing to clients.
(other)live OHLC, turnover, volume, market cap — used by /api/nepse/market not TV

Curated mappings (migration 000018_add_tv_symbol backfill, poller tvSymbolForIndex / tvSymbolForSubIndex):

Indextv_symbol
NEPSE IndexNEPSE
Sensitive IndexSENSITIVE
Float IndexFLOAT
Sensitive Float IndexSFLOAT
BankingBANKING
Development BankDEVBANK
FinanceFINANCE
Hotels And TourismHOTELS
HydroPowerHYDRO
InvestmentINVEST
Life InsuranceLIFEINS
Manu.& Pro.MFG
MicrofinanceMICROFIN
Mutual FundMUTUAL
Non Life InsuranceNONLIFE
OthersOTHERS
TradingTRADING

Any future index NEPSE adds gets a deterministic slug via slugifyTVSymbol (uppercase, alnum-only, 16-char cap) so the NOT NULL upsert never fails.

Daily EOD rows for indices and sub-indices land in nepse_index_history (~6500 rows, since 2012) and nepse_sub_index_history.

ColumnTypeNotes
index_id / sub_index_iduuidFK to master
trade_datedateEOD date
open, high, low, closenumeric(12,2)
volumebigint nullable

Same storage model as stocks: daily is permanent; weekly/monthly are aggregated on demand with date_trunc('week', ...) / date_trunc('month', ...) in the same bucketing query shape.

Resolution → table routing

Given a TV resolution string, the service picks one of three paths:

Reference: service.goGetHistory.

The same flow runs for every kind. After resolve() identifies the ticker as kindStock, kindIndex, or kindSubIndex, the resolution switch picks the same-shape helper from the right table family (nepse_price_history vs. nepse_index_history vs. nepse_sub_index_history; nepse_intraday_prices vs. nepse_intraday_indices vs. nepse_intraday_sub_indices). The retention-window check, nextTime hint, and OHLCV marshalling are shared.

Symbol lookup flow

/symbols is the only TV endpoint backed by a persistent Redis cache. The full lookup:

  • Cache TTL: 5 minutes (cache.TTLNepseTVSymbol())
  • Key: nepse:tv:symbol:<UPPER_SYMBOL>
  • Redis unavailability is non-fatal: a cache read error logs a warning and falls through to the DB; a cache write failure logs a warning but still returns the resolved symbol.

Search flow

/search goes straight to Postgres — no Redis caching because queries are highly variable (every keystroke from the frontend).

The SQL prefix-boost makes NAB return NABBC, NABIL, NABILD2089, NABILD87 (symbol-prefix matches) before name-substring matches like KDBY (which contains "nab" inside Dhanabriddhi Yojana). Typing ^ alone surfaces every index and sub-index; typing ^NEP returns ^NEPSE; typing BANK returns ^BANKING, ^DEVBANK, and any company ticker containing BANK.

-- Postgres forbids expression-based ORDER BY on a bare UNION, so the three
-- branches are wrapped in a subquery. The outer ORDER BY then runs CASE
-- against the merged column set.
SELECT tv_symbol, description, kind
FROM (
    (
        SELECT symbol AS tv_symbol, name AS description, 'stock'::text AS kind
        FROM nepse_companies
        WHERE status = 'A'
          AND (symbol ILIKE '%' || @search || '%' OR name ILIKE '%' || @search || '%')
    ) UNION ALL (
        SELECT '^' || tv_symbol AS tv_symbol, index_name AS description, 'index'::text AS kind
        FROM nepse_indices
        WHERE tv_symbol ILIKE '%' || @search || '%'
           OR index_name ILIKE '%' || @search || '%'
           OR ('^' || tv_symbol) ILIKE '%' || @search || '%'
    ) UNION ALL (
        SELECT '^' || tv_symbol AS tv_symbol, name AS description, 'index'::text AS kind
        FROM nepse_sub_indices
        WHERE tv_symbol ILIKE '%' || @search || '%'
           OR name ILIKE '%' || @search || '%'
           OR ('^' || tv_symbol) ILIKE '%' || @search || '%'
    )
) AS merged
ORDER BY
  CASE WHEN tv_symbol ILIKE @search || '%' THEN 0 ELSE 1 END,
  tv_symbol
LIMIT @lim;

Where live ticks fit in (future)

/history ends where the Postgres database ends. Live bars currently forming during market hours are only visible after the poller's next flush — typically within a minute.

For v1, the frontend uses TV's default UDFCompatibleDatafeed, which polls /history every ~10s for the current bar. Bars update at poll cadence — perceptually fine for OHLC candles, and nginx caching plus the Go app's rate limiter absorb the traffic easily at a few hundred concurrent chart viewers.

The parked Phase 6 plan (see docs/TRADINGVIEW_UDF_PHASE6.md) adds a WebSocket gateway for tick-level latency. Decision: deferred until measured pain justifies the build — laggy charts, polling bottleneck on /history, or >500 concurrent chart viewers. See Integration for the frontend seam when it's time to wire it up.

Design rule: never push bar aggregation logic into the WebSocket gateway. Send raw ticks to the browser; aggregate in the datafeed's subscribeBars callback per-resolution. Server-side aggregation per-subscriber-per-resolution wastes CPU.

Freshness guarantees

EndpointFreshness
/configChanges only on redeploy. In-process cache is populated at first call.
/timeWall-clock accurate — returns time.Now().UTC().Unix() on every call.
/symbolsCached in Redis for 5 minutes. A new listing appears after TTL expires or when the cleanup cron invalidates.
/searchReads directly from nepse_companies — as fresh as the last company-sync job.
/history intradayTrails live by at most one poll cycle (seconds, not minutes).
/history dailyUpdated after the EOD cron at 15:15 NPT Mon–Fri.
/history weekly/monthlyAggregated on every request from current daily data.