MongoDB与未知的findOne方法造成大的延迟 – New Relic
我设置newrelic更好地了解我的应用程序有什么瓶颈，我发现一个问题，我似乎无法弄清楚。
我的延迟大部分是由mongoDB user.fineOne引起的，但主要的问题是，我似乎无法find代码中发生的地方。
在下面的图片中，您可以看到我的API调用get/all/proposal结束点的跟踪详细信息。 前14个方法调用是我的server.js中的中间件，之后它是一个中间件：在MongoDB用户findOne中进行 身份validation，在那里是延迟。
获取/所有/build议的代码：
app.get('/all/proposals',isLoggedIn,function(req, res) { Proposal.find().sort({proposalNo: -1}).limit(5).exec(function(err,proposal){ if(err){ console.log(err); }else{ console.log("All Proposals " + proposal); res.json(proposal); } }); });
现在在任何时候，我都可以看到我在get/all/proposals上在MongoDB上运行User.findOne调用 。 最初我以为是isLoggedIn中间件，我检查用户是否在会话（Passport.js），但正如你所看到的isLoggedIn Middleware只需要0.0222（毫秒）。
同样的问题出现在多个API端点上，即get/caseStudy和它总是user.findOne另一个例子如下：
谁能帮我解决这个问题吗？ 请让我知道，如果你需要更多的细节，我猜你会。
更新： Server.js代码
// set up ====================================================================== require('newrelic'); var express = require('express'); var app = express(); // create our app w/ express var server = require('http').createServer(app); var mongoose = require('mongoose'); // mongoose for mongodb var port = process.env.PORT || 8080; // set the port var database = require('./config/db'); // load the database config var morgan = require('morgan'); // log requests to the console (express4) var bodyParser = require('body-parser'); // pull information from HTML POST (express4) var methodOverride = require('method-override'); // simulate DELETE and PUT (express4) var passport = require('passport'); var flash = require('connect-flash'); var session = require('express-session'); var cookieParser = require('cookie-parser'); var compression = require('compression'); var nodemailer = require('nodemailer'); var busboy = require("connect-busboy"); // configuration =============================================================== mongoose.connect(database.url); // connect to mongoDB database on modulus.io require('./config/passport')(passport); app.use(express.static(__dirname + '/public')); app.use(express.static(__dirname + '/views')); // set the static files location /public/img will be /img for users app.use(busboy()); app.use(compression()); //use compression app.use(morgan('dev')); // log every request to the console app.use(bodyParser.urlencoded({'extended': true})); // parse application/x-www-form-urlencoded app.use(bodyParser.json()); // parse application/json app.use(bodyParser.json({ type: 'application/vnd.api+json' })); // parse application/vnd.api+json as json app.use(methodOverride()); app.use(cookieParser()); // read cookies (needed for auth) app.set('view engine', 'ejs'); // set up ejs for templating // required for passport app.use(session({ secret: '', resave: false, saveUninitialized: false })); // session secret app.use(passport.initialize()); app.use(passport.session()); // persistent login sessions app.use(flash()); // use connect-flash for flash messages stored in session // routes ====================================================================== require('./routes/index.js')(app, passport); // load our routes and pass in our app and fully configured passport //require('./routes/knowledgeBase/index.js')(app, passport); require('./routes/bios/index.js')(app, passport); // listen (start app with node server.js) ====================================== app.listen(port); console.log("App listening on port " + port);
更新2： Passport.js
// config/passport.js // load all the things we need var LocalStrategy = require('passport-local').Strategy; var crypto = require("crypto"); var api_key = ''; var domain = ''; var mailgun = require('mailgun-js')({apiKey: api_key, domain: domain}); // load up the user model var User = require('../app/models/user'); // expose this function to our app using module.exports module.exports = function(passport) { // ========================================================================= // passport session setup ================================================== // ========================================================================= // required for persistent login sessions // passport needs ability to serialize and unserialize users out of session // used to serialize the user for the session passport.serializeUser(function(user, done) { done(null, user.id); }); // used to deserialize the user passport.deserializeUser(function(id, done) { User.findById(id, function(err, user) { done(err, user); }); }); // ========================================================================= // LOCAL SIGNUP ============================================================ // ========================================================================= // we are using named strategies since we have one for login and one for signup // by default, if there was no name, it would just be called 'local' passport.use('local-signup', new LocalStrategy({ // by default, local strategy uses username and password, we will override with email firstNameField: 'firstName', lastNameField: 'lastName', usernameField: 'email', passwordField: 'password', jobTitleField: 'jobTitle', startDateField: 'startDate', passReqToCallback: true // allows us to pass back the entire request to the callback }, function(req, email, password, done) { // find a user whose email is the same as the forms email // we are checking to see if the user trying to login already exists User.findOne({ 'email': email }, function(err, user) { // if there are any errors, return the error if (err) return done(err); // check to see if theres already a user with that email if (user) { return done(null, false, { message: 'That email is already taken.' }); } else { var token = crypto.randomBytes().toString(); // if there is no user with that email // create the user var newUser = new User(); // set the user's local credentials newUser.firstName = req.body.firstName; newUser.lastName = req.body.lastName; newUser.email = email; newUser.password = newUser.generateHash(password); // use the generateHash function in our user model newUser.jobTitle = req.body.jobTitle; newUser.startDate = req.body.startDate; newUser.birthday = req.body.birthday; newUser.region = req.body.region; newUser.sector = req.body.sector; newUser.accountConfirmationToken = token; newUser.accountConfirmationTokenExpires = Date.now() + 3600000; newUser.accountVerified = 'false'; // save the user newUser.save(function(err) { if (err) throw err; else { return done(null, newUser); } }); } }); })); // ========================================================================= // LOCAL LOGIN ============================================================= // ========================================================================= // we are using named strategies since we have one for login and one for signup // by default, if there was no name, it would just be called 'local' passport.use('local-login', new LocalStrategy({ // by default, local strategy uses username and password, we will override with email usernameField : 'email', passwordField : 'password', passReqToCallback : true // allows us to pass back the entire request to the callback }, function(req, email, password, done) { // callback with email and password from our form // find a user whose email is the same as the forms email // we are checking to see if the user trying to login already exists User.findOne({ 'email' : email }, function(err, user) { // if there are any errors, return the error before anything else if (err) return done(err); // if no user is found, return the message if (!user) return done(null, false, req.flash('loginMessage', 'No user found.')); // req.flash is the way to set flashdata using connect-flash // if the user is found but the password is wrong if (!user.validPassword(password)) return done(null, false, req.flash('loginMessage', 'Oops! Wrong password.')); // create the loginMessage and save it to session as flashdata if(user.accountVerified == 'false') return done(null, false, req.flash('loginMessage', 'Looks like you have not verified your account after registeration.')); else user.lastLogin = Date.now(); user.save(function(err) { if (err) throw err; else { // all is well, return successful user return done(null, user); } }); }); })); };
更新3： isLoggedIn函数
// route middleware to make sure a user is logged in function isLoggedIn(req, res, next) { // if user is authenticated in the session, carry on if (req.isAuthenticated()) return next(); // if they aren't redirect them to the home page res.redirect('/'); }
更新4：获取提议的步骤
第1步：首先加载build议页面
app.get('/proposals',isLoggedIn,function(req, res) { res.render('proposals.ejs', { user : req.user // get the user out of session and pass to template }); });
步骤2：build议页面有一个angular.js控制器/工厂，它在页面加载时调用以下函数来获取数据。
// ========================================================================= // FUNCTIONS TO BE RUN WHEN THE PAGE FIRST LOADS TO POPULATE FRONT-END ===== // ========================================================================= $scope.intialize = function() { $scope.getAllSectors(); $scope.getLatestProposals(); } // =============================== // GET LATEST *5* PROPOSALS ===== // =============================== factory.getLatestProposals = function() { return $http.get('/all/proposals') .then(function(response) { //promise is fulfilled deferred.resolve(response.data); console.log("readched the filtered project service!"); //promise is returned // return deferred.promise; return response.data; }, function(response) { deferred.reject(response); //promise is returned return deferred.promise; }); };
第3步：调用/all/proposals路由
// ======================= // GET All Proposals ===== // ======================= app.get('/all/proposals',isLoggedIn,function(req, res) { Proposal.find().sort({proposalNo: -1}).limit(5).exec(function(err,proposal){ if(err){ console.log(err); }else{ console.log("All Proposals " + proposal); res.json(proposal); } }); });
查看您提供的代码后，您的性能日志中显示的.findOne()似乎是在search用户并对其进行身份validation时执行的。
因此，看起来性能瓶颈发生在以下两个查询之一：
/* * LOCAL LOGIN */ // find a user whose email is the same as the forms email // we are checking to see if the user trying to login already exists User.findOne({ 'email' : email }, function(err, user) { ... /* * LOCAL SIGNUP */ // find a user whose email is the same as the forms email // we are checking to see if the user trying to login already exists User.findOne({ 'email': email ...
我发现在您的两个护照本地策略中，您正在searchemail字段，因此您可以通过在该字段上创build索引来提高性能。
要尝试优化LOCAL LOGIN findOne查询，可以尝试在email字段中为users集合添加索引（如果还没有）：
// This index will optimize queries that search against the {email} field db.users.createIndex({ email: 1});
更新＃1
我发现了一个相关的Stack Overflow主题 ，可能是您的性能问题的答案 – 您应该更新express.jsconfiguration中的以下行：
app.use(session({ secret: '', resave: false, saveUninitialized: false }));
至
app.use(session({ secret: '', resave: true, saveUninitialized: true }));
我还设法在Express JS文档中find有关saveUninitalized和saveUninitalized属性的说明：
saveUninitialized
强制将“未初始化”的会话保存到商店。 会话在新build但未修改时未初始化。 selectfalse对于实现login会话，降低服务器存储使用率或在设置cookie之前遵守需要许可的法律很有用。 select错误也有助于客户在没有会话的情况下进行多个并行请求的竞争条件。
默认值为true，但使用默认值已被弃用，因为默认值将在以后更改。 请研究这个设置，并select适合您的用例。
注意，如果您将Session与PassportJS结合使用，则Passport将在会话中添加一个空的Passport对象，以供用户validation之后使用，这将作为会话的修改处理，从而导致将其保存起来。 这已在PassportJS 0.3.0中修复
重新保存
强制会话保存回会话存储区，即使请求期间会话永远不会被修改。 根据您的商店，这可能是必要的， 但它也可以创build竞争条件，其中一个客户端向您的服务器发出两个并行请求，并且一个请求中的会话更改可能会在另一个请求结束时被覆盖，即使它没有改变 （这种行为也取决于你正在使用的商店）。
默认值为true，但使用默认值已被弃用，因为默认值将在以后更改。 请研究这个设置，并select适合您的用例。 通常情况下，你会想要假。
我怎么知道这是否是我的商店所必需的？ 最好的方法是检查你的商店，如果它实现了触摸方法。 如果是这样，那么你可以安全地设置resave：false。 如果它没有实现触摸方法，并且商店在存储的会话上设置了到期date，那么您可能需要resave：true。