Materialized Views with MongoDB

Danilo Assis

Danilo Assis

daniloab_daniloab
Woovi Logo

Tech Lead

Overview

  • Introduction
  • What are materialized views?
  • Comparison with Standard Views
  • Indexes and Performance
  • How do materialized views work?
  • Benefits of using materialized views
  • Use cases for materialized views in MongoDB
  • $merge and $out stage
  • Examples
  • Best practices for using materialized views
  • Conclusion

Introduction

  • Learn
  • Purpose
  • Knowledge
  • Start to use
  • Team will increase

What are materialized views?

  • Database Objects
  • Results of Precomputed Query
  • Differences from Regular Views

Comparison with Standard Views

  • MongoDB offer both
  • Materialized is saved on disk
  • $merge or $out stage to update the saved data
  • both results from aggregation pipeline

Indexes and Performance

  • Standard use indexes from underlying collection
  • Materialized can has it own indexes
  • Materialized allow faster access to data reading from disk

How do materialized views work?

  • aggregate performs the calc
  • $merge writes the output, update matched previous results
  • Regularly updates
  • Jobs

Benefits of using materialized views

  • Deliver high-quality software
  • Allow faster access to data
  • Benefits in data privacy
  • Increase of performance
  • Split in databases
  • Simplify complex queries
  • Big O of queries

Use cases for materialized views in MongoDB

  • Complex analytics queries
  • Improve performance
  • Support for real time dashboards
  • Cache frequently accessed data
  • Abstract data from many collections/tables into only one

$merge and $out stage

  • Can output to a collection in the same or different database.
  • Creates a new collection if the output collection does not already exist.
  • Can incorporate results (insert new documents, merge documents, replace documents, keep existing documents, fail the operation, process documents with a custom update pipeline) into an existing collection.
  • Can output to a sharded collection. Input collection can also be sharded.
  • Merge save/update a result in a collection
  • Out save/replace a result in a collection

Merge and Out Comparison

Materialized Views with Merge Stage

db.Charge.aggregate([
  {
    $group: {
      _id: { $dateToString: { format: "%Y-%m-%d", date: "$paidAt" } },
      totalValue: { $sum: "$valueWithDiscount" },
    },
  },
  {
    $merge: {
      into: "ChargeByDay",
      on: "_id",
      whenMatched: "merge",
      whenNotMatched: "insert",
    },
  },
]);

Materialized Views with Out Stage

db.Charge.aggregate([
  {
    $group: {
      _id: { $dateToString: { format: "%Y-%m-%d", date: "$paidAt" } },
      totalValue: { $sum: "$valueWithDiscount" },
    },
  },
  {
    $out: "ChargeByDay",
  },
]);

Best practices for using materialized views

  • Updated regularly
  • Careful with pipeline stages
  • Good use of indexes with group stage
  • Chunk Size
  • Shard Key

Drawbacks of using materialized views

  • Increased Storage
  • Write Latency
  • Increased complexity
  • Stale data

Thanks

We are hiring!

woovi.com/jobs