Debug Mode

Debug mode provides detailed logging and diagnostic information for troubleshooting SupaScan indexing, API, and database issues.

Enabling Debug Mode

Environment Variables

# Enable debug logging
export LOG_LEVEL=debug
export NODE_ENV=development
export SUPASCAN_DEBUG=true

# Run SupaScan
npm start

In .env File

LOG_LEVEL=debug
NODE_ENV=development
SUPASCAN_DEBUG=true
CLICKHOUSE_DEBUG=true
INDEXING_DEBUG=true

Runtime Configuration

// Enable debug at runtime
import { logger } from './utils/logger';
import { config } from './config';

config.debug = true;
logger.level = 'debug';

Debug Output

Log Levels

Level	Description	Use Case
error	Critical errors	System failures
warn	Warnings	Potential issues
info	Information	Normal operations
debug	Debug details	Development/troubleshooting
silly	Everything	Deep debugging

Debug Information Examples

[2024-01-15 10:23:45] DEBUG: Processing block 12345678
  slot: 12345678
  transactions: 45
  tokenTransfers: 23
  dexSwaps: 12
  processingTime: 234ms
  clickhouseInsertTime: 12ms

[2024-01-15 10:23:46] DEBUG: API Request
  endpoint: /v1/tokens/So111.../analysis
  userId: user_123
  responseTime: 89ms
  cacheHit: false

Debug Commands

API Debug Endpoints

GET /debug/status

Shows current system status:

{
  "debugMode": true,
  "logLevel": "debug",
  "environment": "development",
  "memoryUsage": "124MB / 512MB",
  "uptime": "2h 34m",
  "indexingStatus": "active",
  "clickhouseConnections": 8,
  "queueSize": 3
}

GET /debug/indexing

Indexing performance metrics:

{
  "blocksProcessed": 12345,
  "blocksPerSecond": 2.3,
  "lastProcessedSlot": 12345678,
  "currentSlot": 12345680,
  "lagSeconds": 2,
  "failedBlocks": 3,
  "averageProcessingTime": "234ms"
}

GET /debug/clickhouse

ClickHouse cluster status:

{
  "clusterHealth": "healthy",
  "activeConnections": 8,
  "queryCount": 1234,
  "averageQueryTime": "12ms",
  "slowQueries": 2,
  "memoryUsage": "2.1GB",
  "diskUsage": "45GB"
}

GET /debug/api

API performance metrics:

{
  "totalRequests": 5678,
  "averageResponseTime": "89ms",
  "p95ResponseTime": "234ms",
  "errorRate": "0.5%",
  "cacheHitRate": "87%",
  "rateLimitHits": 12
}

Debug Features

Indexing Debug

// Block processing debug
class IndexingDebugger {
  async processBlock(slot: number) {
    const start = Date.now();
    logger.debug('Processing block', { slot });
    
    try {
      const block = await this.fetchBlock(slot);
      logger.debug('Block fetched', { 
        slot, 
        transactions: block.transactions?.length || 0,
        fetchTime: Date.now() - start 
      });
      
      const parsed = await this.parseBlock(block);
      logger.debug('Block parsed', {
        slot,
        tokenTransfers: parsed.tokenTransfers.length,
        dexSwaps: parsed.dexSwaps.length,
        parseTime: Date.now() - start
      });
      
      await this.insertToClickHouse(parsed);
      logger.debug('Block inserted', {
        slot,
        totalTime: Date.now() - start
      });
    } catch (error) {
      logger.error('Block processing failed', { slot, error: error.message });
    }
  }
}

ClickHouse Query Debugging

// ClickHouse query logging
class ClickHouseDebugger {
  async query(sql: string, params?: any[]) {
    const start = Date.now();
    logger.debug('ClickHouse Query', { 
      sql: sql.substring(0, 200) + '...',
      params: params?.length || 0
    });
    
    try {
      const result = await this.client.query({ query: sql, params });
      logger.debug('ClickHouse Response', {
        duration: Date.now() - start,
        rows: result.rows,
        bytes: result.bytes
      });
      return result;
    } catch (error) {
      logger.error('ClickHouse Error', {
        sql: sql.substring(0, 200),
        error: error.message,
        duration: Date.now() - start
      });
      throw error;
    }
  }
}

API Request Debugging

// API request tracing
app.use((req, res, next) => {
  const requestId = generateRequestId();
  req.requestId = requestId;
  
  logger.debug('API Request', {
    requestId,
    method: req.method,
    url: req.url,
    userAgent: req.get('User-Agent'),
    ip: req.ip
  });
  
  const start = Date.now();
  
  res.on('finish', () => {
    logger.debug('API Response', {
      requestId,
      statusCode: res.statusCode,
      duration: Date.now() - start
    });
  });
  
  next();
});

Debug Tools

ClickHouse Query Analysis

-- Monitor slow queries
SELECT 
    query,
    query_duration_ms,
    read_rows,
    read_bytes,
    memory_usage
FROM system.query_log
WHERE event_date = today()
  AND query_duration_ms > 1000
ORDER BY query_duration_ms DESC;

-- Check table sizes
SELECT 
    table,
    formatReadableSize(sum(bytes)) as size,
    count() as parts
FROM system.parts
WHERE database = 'supascan'
GROUP BY table
ORDER BY sum(bytes) DESC;

Indexing Performance

# Monitor indexing lag
curl http://localhost:3000/debug/indexing

# Check RPC node health
curl -X POST https://api.mainnet-beta.solana.com \
  -H "Content-Type: application/json" \
  -d '{"jsonrpc":"2.0","id":1,"method":"getHealth"}'

# Test ClickHouse connection
curl 'http://localhost:8123/?query=SELECT%201'

Memory and CPU Profiling

# Start with profiling
node --inspect --max-old-space-size=4096 src/index.js

# Connect Chrome DevTools
chrome://inspect

# Monitor memory usage
node --max-old-space-size=4096 --trace-gc src/index.js

Debug Configuration

Development Config

// config/debug.ts
export const debugConfig = {
  // Logging
  logLevel: 'debug',
  logIndexing: true,
  logClickHouse: true,
  logAPI: true,
  logWebhooks: true,
  
  // Performance
  profileMemory: true,
  trackMetrics: true,
  slowQueryThreshold: 1000,
  
  // Development
  prettyPrint: true,
  stackTraces: true,
  verboseErrors: true
};

Debug Middleware

// API error handling
app.use((err, req, res, next) => {
  if (config.debug) {
    logger.error('API Error', {
      error: err,
      stack: err.stack,
      requestId: req.requestId,
      url: req.url,
      method: req.method
    });
    
    res.status(500).json({
      error: 'Internal Server Error',
      debug: err.stack,
      requestId: req.requestId
    });
  } else {
    logger.error('API Error', { error: err.message });
    res.status(500).json({ error: 'Internal Server Error' });
  }
});

Common Debug Scenarios

1. Indexing Lag

# Check indexing status
curl http://localhost:3000/debug/indexing

# Check RPC node health
curl -X POST https://api.mainnet-beta.solana.com \
  -H "Content-Type: application/json" \
  -d '{"jsonrpc":"2.0","id":1,"method":"getHealth"}'

# Check ClickHouse cluster
curl 'http://localhost:8123/?query=SELECT%20hostName(),%20uptime()%20FROM%20system.clusters'

2. ClickHouse Issues

-- Check cluster health
SELECT hostName(), uptime(), version() FROM system.clusters;

-- Check replication status
SELECT table, replica_name, is_leader, log_pointer 
FROM system.replicas 
WHERE database = 'supascan';

-- Monitor slow queries
SELECT query, query_duration_ms, read_rows 
FROM system.query_log 
WHERE event_date = today() 
ORDER BY query_duration_ms DESC 
LIMIT 10;

3. API Performance Issues

# Check API metrics
curl http://localhost:3000/debug/api

# Test specific endpoint
curl -H "Authorization: Bearer sk_test_123" \
  http://localhost:3000/v1/tokens/So111.../analysis

# Check rate limiting
curl -H "Authorization: Bearer sk_test_123" \
  http://localhost:3000/v1/debug/rate-limits

4. Memory and Performance

// Memory monitoring
setInterval(() => {
  const usage = process.memoryUsage();
  logger.debug('Memory snapshot', {
    rss: Math.round(usage.rss / 1024 / 1024),
    heap: Math.round(usage.heapUsed / 1024 / 1024),
    external: Math.round(usage.external / 1024 / 1024)
  });
}, 60000);

// Performance timing
class PerformanceDebugger {
  private timers = new Map();
  
  start(label: string) {
    this.timers.set(label, Date.now());
    logger.debug(`Timer started: ${label}`);
  }
  
  end(label: string) {
    const start = this.timers.get(label);
    if (start) {
      const duration = Date.now() - start;
      logger.debug(`Timer ended: ${label}`, { duration });
      this.timers.delete(label);
    }
  }
}

Debug Best Practices

1. Use Structured Logging

   // Good
   logger.debug('Block processed', { slot, transactions, duration });
   
   // Avoid
   console.log(`Block ${slot} processed with ${transactions} txs`);
   

2. Add Context to Logs

   logger.debug('Indexing failed', {
     context: 'block_processing',
     slot: 12345678,
     error: err.message,
     retryCount: 3
   });
   

3. Use Debug Namespaces

   const debug = {
     indexing: require('debug')('supascan:indexing'),
     clickhouse: require('debug')('supascan:clickhouse'),
     api: require('debug')('supascan:api')
   };
   

4. Monitor Performance

   // Track slow operations
   if (duration > config.slowQueryThreshold) {
     logger.warn('Slow operation', { operation, duration });
   }
   

5. Clean Up Debug Code

   if (process.env.NODE_ENV !== 'production') {
     // Debug only code
   }
   

Security Considerations

⚠️ Warning: Debug mode can expose sensitive information!

• Never enable in production
• Don't log API keys or private data
• Sanitize user input in logs
• Restrict debug endpoints to admins
• Clear debug logs regularly
• Use HTTPS for debug endpoints

---