A progressive Node.js framework for building efficient and scalable server-side applications, heavily inspired by Angular.
Nest framework TypeScript starter repository.
$ npm install# development
$ npm run start
# watch mode
$ npm run start:dev
# production mode
$ npm run start:prod# development
$ npm run build:app# unit tests
$ npm run test
# e2e tests
$ npm run test:e2e
# test coverage
$ npm run test:covThis application can be built using a microservice architecture such that all modular system are built as separate service while having the logManager as a single service system that house the routing strategy to the other microservices system.
Considering the kind of architecture. each service will either have a unique route pattern such as `users/`, `adminitrattions/` `payments/`, these patterns can be used to denote each service system and able to detect which log activities belongs to a particular microservice or having a unique serviceId present in every request header of that service.
In this case, I will be using a structural logging over file system in order to enable ease in data querying and mining.
Propose the tools that can be used to build that application taking into account the following challenges of data: variety, velocity, and volume.
There are several log management tools such as
1. Loggly
2. Logstash
3. Splunk
4. PaperTrails
5. Storm
6. Graylog2
The major overall issues is in
1. volume of data
2. Price
3. Security regarding sensitive data.
These tools are capable in logging resources but there are one or more issues to lookout for.
1. Loggly: This is a tools mainly for devops to parse data coming from your app servers. Anything beyond that you’ll have to build yourself.
2. Logstash: This is stack because it entails one or more technology such as kibana and elastic search. these stacks requires configuring separate instance on production which brings more management and complexity
3. Splunk: For high scalability, requires dedicated cluster and somewhat subjective which means the cost of running it is high and it is also complex coming to management.
4. PaperTrails: This is mosty a text based logging system which gives complexity for proper and advance data mining.
5. Storm: This is limiting in amount of data that can be sent.
6. Graylog2: The newest among others, this makes it limiting in usage.
The only alternative tools I think is best if we are more concerned about high level of customization and proprietary changes is `Build it yourself` and I would suggest the use of
1. Document DB such MongoDB
2. Elastic Search for deep querying and analysis.
1 I adopted the use of `serviceId` as an identify for each microservice
2. This `serviceId` is predefined ID for each micro service and these are setup in env variables and each pointing to their baseurl.
3. I then make available a route strategy to pass request to the expected microservice
4. The interceptor is used to carry the main processing of logging by using a cloud support `Loggly`.
5. Configuration params for loggly can be found in env
6. This interceptor helps to log request after a complete cycle to loggly and able to catch all errors that might occur.
1. Implementation of node clustering to enable efficiency for concurrent request scaling.
Due to time and load of work at work. I took less than 24hrs to build to this stage of the app.
So, I was unable to do the followings.
1. Docker for containerization.
2. Node Clustering Implementation.
3. Test Cases.
Nest is an MIT-licensed open source project. It can grow thanks to the sponsors and support by the amazing backers. If you'd like to join them, please read more here.
- Author - Kamil Myśliwiec
- Website - https://nestjs.com
- Twitter - @nestframework
Nest is MIT licensed.