Which is the fastest way to remove MongoDB documents by Date? -
in our company have retention of 8 days of data (with 1 million of records aprox.) have cronjob remove documents older 8 days each day. we're using published field , this field not indexed.
it takes 15 minutes finish rid off 100.000 records , found operation long.
this query 'docs' variable array of documents don't want remove. 'thedate' variable date of 8 days ago.
records.remove( { "published" : { $lte : thedate }, "_id" : { $nin : docs } } would better use _id field, indexed, in ordered operation? how can use _id field in order same operation?
discard cron job entirely: job ttl indexes. http://docs.mongodb.org/manual/core/index-ttl/
create ttl index on published field expireafterseconds: 691200 , watch documents automatically removed 8 days after publication.
and if don't want indiscriminately delete documents 8 days after publication, keep cron job , create plain index on published field.
Comments
Post a Comment