使用.limit()和Firestore查询
我有一个使用Firestore作为队列的应用程序。 数据来自另一个来源,存储在Firestore的/data/{id}
, unprocessed
的属性设置为true。 在我的Node.js脚本中,我想要一次查询这些未处理的logging,以缓慢地处理它们。 有成千上万的logging,所以试图将它们全部加载到内存中会使进程崩溃。
码:
firestore.collection('data').where('unprocessed', '==', true).limit(25).onSnapshot((snapshot) => { snapshot.forEach((doc) => { processItem(doc); }); });
processItem()
函数执行必要的处理,将数据保存回Firestore, unprocessed
属性设置为false。
我正在运行的问题是,任何时候我试图运行这个代码,我得到以下错误:
Error: Error 3: Order must include __name__ at sendError (C:\[myApp]\node_modules\@google-cloud\firestore\src\watch.js:254:15) at DestroyableTransform.stream.on.proto (C:\[myApp]\node_modules\@google-cloud\firestore\src\watch.js:532:13) at emitOne (events.js:115:13) at DestroyableTransform.emit (events.js:210:7) at addChunk (C:\[myApp]\node_modules\readable-stream\lib\_stream_readable.js:284:12) at readableAddChunk (C:\[myApp]\node_modules\readable-stream\lib\_stream_readable.js:271:11) at DestroyableTransform.Readable.push (C:\[myApp]\node_modules\readable-stream\lib\_stream_readable.js:238:10) at DestroyableTransform.Transform.push (C:\[myApp]\node_modules\readable-stream\lib\_stream_transform.js:146:32) at afterTransform (C:\[myApp]\node_modules\readable-stream\lib\_stream_transform.js:102:51) at TransformState.afterTransform (C:\[myApp]\node_modules\readable-stream\lib\_stream_transform.js:79:12) at DestroyableTransform.noop [as _transform] (C:\[myApp]\node_modules\through2\through2.js:26:3) at DestroyableTransform.Transform._read (C:\[myApp]\node_modules\readable-stream\lib\_stream_transform.js:182:10) at DestroyableTransform.Transform._write (C:\[myApp]\node_modules\readable-stream\lib\_stream_transform.js:170:83) at doWrite (C:\[myApp]\node_modules\readable-stream\lib\_stream_writable.js:406:64) at writeOrBuffer (C:\[myApp]\node_modules\readable-stream\lib\_stream_writable.js:395:5) at DestroyableTransform.Writable.write (C:\[myApp]\node_modules\readable-stream\lib\_stream_writable.js:322:11)
从我的代码中删除.limit()
使它运行,但后来我遇到了另一个问题,处理和存储项目的function没有正确完成,我的内存使用只是不断增长,直到进程崩溃。
我的第一个直觉是认为.limit()
与.onSnapshot()
出于某种原因不兼容,但也许有人可以给我一个更好的想法在这里发生了什么。
编辑
我还尝试添加.orderBy('__name__')
,如https://firebase.google.com/docs/firestore/manage-data/delete-data (位于“删除集合”部分)中所示,但这只是导致另一个错误:
Error: Trying to compare documents on fields that don't exist. Please include the fields you are ordering on in your select() call. at C:\[myApp]\node_modules\@google-cloud\firestore\src\reference.js:1679:19 at Array.sort (native) at computeSnapshot (C:\[myApp]\node_modules\@google-cloud\firestore\src\watch.js:438:20) at push (C:\[myApp]\node_modules\@google-cloud\firestore\src\watch.js:469:18) at DestroyableTransform.stream.on.proto (C:\[myApp]\node_modules\@google-cloud\firestore\src\watch.js:514:15) at emitOne (events.js:115:13) at DestroyableTransform.emit (events.js:210:7) at addChunk (C:\[myApp]\node_modules\readable-stream\lib\_stream_readable.js:284:12) at readableAddChunk (C:\[myApp]\node_modules\readable-stream\lib\_stream_readable.js:271:11) at DestroyableTransform.Readable.push (C:\[myApp]\node_modules\readable-stream\lib\_stream_readable.js:238:10) at DestroyableTransform.Transform.push (C:\[myApp]\node_modules\readable-stream\lib\_stream_transform.js:146:32) at afterTransform (C:\[myApp]\node_modules\readable-stream\lib\_stream_transform.js:102:51) at TransformState.afterTransform (C:\[myApp]\node_modules\readable-stream\lib\_stream_transform.js:79:12) at DestroyableTransform.noop [as _transform] (C:\[myApp]\node_modules\through2\through2.js:26:3) at DestroyableTransform.Transform._read (C:\[myApp]\node_modules\readable-stream\lib\_stream_transform.js:182:10) at DestroyableTransform.Transform._write (C:\[myApp]\node_modules\readable-stream\lib\_stream_transform.js:170:83)
如果您添加.orderBy(FirebaseFirestore.FieldPath.documentId())
,它应该解决问题。 这里解释为什么这个问题解决了这个问题。 这是一个例子的要点 。