#Issue17
3 posts

How Sqreen handles 50,000 requests every minute in a write-heavy environment

Caching to handle large volumes of requests. Using queues to reduce the number of write operations demanded within a given time frame to guard against overload.
Read more

How Sqreen handles 50,000 requests every minute in a write-heavy environment

Three principles used by Sqreen to build this scalable architecture:

  • Caching to handle large volumes of requests.
  • Using queues to reduce the number of write operations demanded within a given time frame to guard against overload.
  • Designing applications in a way that scalability is achieved by using more or fewer iterations of the same module to scale up and down, rather than requiring a single unit to work faster or harder.

Full post here, 9 mins read

Site reliability engineering best practices for data pipelines

Reduce hot-spotting by balancing out the workload across resources. Utilize autoscaling. Adhere to strict access control for privacy, security and data integrity.
Read more

Site reliability engineering best practices for data pipelines

  • Define and measure service-level objectives (SLOs) to ensure data freshness, data correctness, and superior data isolation.
  • Plan for dependency failures by checking for overdependence on products that don’t meet their own SLOs.
  • Create and maintain system diagrams, process documentation and playbook entries that outline recovery from alert conditions.
  • Reduce hot-spotting by balancing out the workload across resources.
  • Utilize autoscaling.
  • Adhere to strict access control for privacy, security and data integrity.
  • Use idempotent and two-phase mutations to avoid duplication or storage of incorrect data in case of pipeline failure in the middle of a process.
  • Use checkpointing to store partially implemented processes.

Full post here, 5 mins read

9 Kubernetes security best practices everyone must follow

Always use the latest stable version. Enable role-based access control (RBAC). Avoid granting cluster-wide permissions.
Read more

9 Kubernetes security best practices everyone must follow

  1. Always use the latest stable version.
  2. Enable role-based access control (RBAC). Avoid granting cluster-wide permissions.
  3. Make sure to create non-default namespaces with security boundaries customized by their workload.
  4. Run especially sensitive workloads on a dedicated set of machines to limit fallout from any breaches.
  5. Carefully secure access to cloud metadata.
  6. Create and define cluster network policies.
  7. Define a cluster-wide Pod Security Policy for how workloads may run in each cluster.
  8. Harden node security by ensuring your host is secure & controlling network access to sensitive ports.
  9. Enable audit logs. Actively monitor them to identify access attempts.

Full post here, 5 mins read