Service Base Feature Checklist

Verision Reporting

  • report version dynamically
  • report version in a text file in the deployment directory — if applicable
  • report version inside the .war file — probably in the Manifest
  • embed version in any generated client-facing pages

Logging

  • provide a trace ID either generated or from header for each log message to correlate a thread’s activities
  • for monitoring & alerting, use levels as
    • INFO *sparse* informative messages e.g. one-time initialization, periodic milestones, cycle start/stop, periodic metrics
    • WARN events to watch over time or that might bear investigation e.g. client errors, retries,
    • ERROR a condition that *requires* someone to look at and evaluate; if you can ignore it, it’s a WARN
    • CRITICAL a system failure that needs immediate resolution

Monitoring

  • provides an “alive” URL for monitoring
  • provide URL to validate deeper monitoring (“healthy” internal processes)
  • provide URL to validate *connectivity* remote sytem dependencies
  • Note that we just care about connectivity here. The remote system should have its own health monitor.
  • provide URL for report of error conditions, current and past
  • report errors (push) to a remote system (in addition to logging)
  • for monitoring & alerting, use levels as
    • INFO *sparse* informative messages e.g. one-time initialization, periodic milestones, cycle start/stop, periodic metrics
    • WARN events to watch over time or that might bear investigation e.g. client errors, retries,
    • ERROR a condition that *requires* someone to look at and evaluate; if you can ignore it, it’s a WARN
    • CRITICAL a system failure that needs immediate resolution
  • a good rule of thumb is, if it sets off someone’s pager, it’s ERROR, otherwise it’s a WARN

Runtime Metrics

  • identify tasks. Each task will have an elapsed time, and at least one count associated with it
  • provide running counts & elapsed time measurements of important system functions
  • provide running counts of error conditions
  • provide running counts aggregated into categories based on processing data
  • provide in-memory aggregation & statistics about key running counts
  • system-wide count of errors and warns

Configuration

  • a single mechanism for configuration values, documentation, and ingestion/usage so they don’t get out of sync
  • Don’t repeat yourself!

Documentation

  • documentation in the code, not separate. Don’t repeat yourself!
  • generated API documentation
  • generated configuration documentation
  • generated metrics documentation
  • generated logging documentation

LifeCycle Management

  • All instantiated objects should have lifecycle managed by the application context — create, initialize, destroy
  • Should be easy to hook objects into the application context lifecycle, either decoratively or through discovery.
  • Note: things like object pools or execution pools need to be shutdown gracefully and cleaned up!

Runtime Asynchronous Processing

  • WIP the system needs a plan for how to handle asynchronous processing in a way that’s standard throughout the system.
    • e.g. JMS Queue? Execution thread pool? Batch from database table? In-memory? Persistent?

Offline Batch Processing

  • standard mechasim for parallelizing offline tasks by breaking tasks into batches that can be distributed across machines

State Management / Distributed Caching

  • WIP the system should have a highly available, horizontally scalable mechanism for saving processing state for longer transactions (such as user sessions or inputs to a long-lived messaging conversation)
    • e.g. external (memchached), internal with broadcasting (ehcache), NoSql, relational, in-memory, persistent
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s