image
author

William Dawson

Full Stack Developer

Dotenv integration with NestJS and TypeORM.

While using third party sources in application development, there must be some involvement of SSH keys or API credentials. This becomes a problem when a project is handled by a team of developers. Thus, the source code has to be pushed to git repositories periodically. Once the code is pushed to a repository, anyone can see it with the third-party keys.

A very prominent and widely used solution for this problem is using environment variables. These are the local variables containing some useful information like API keys and are made available to the application or project.

A tool known as dotenv has made it easy to create such variables and making these variables available to the application. It is an easy to use tool which can be added to your project by using any package manager.

We will use yarn as a package manager.

First, add the package using terminal.


yarn add dotenv

Since we are using NestJS which is based on typescript, so we need to add the “@types” package for the same that acts as an interface between javascript and typescript package.


yarn add @types/dotenv

Since the database to be used is Postgres, so install the necessary driver for Postgres.


yarn add pg

Now install the TypeORM module to your nest project.


yarn add @nestjs/typeorm typeorm

Now, create TypeORM entities in your project folder- For this illustration, we will be creating a folder ‘db‘ inside the ‘src‘ folder of our nest project and inside this folder, create another folder ‘entities‘ and create a typescript file containing information about your TypeORM entity.

For the sake of simplicity, we will create a user-entity file. Also, we will be creating an ‘id‘ field, a ‘name‘ field and an ‘email‘ field for this entity.

#src/db/entities/user.entity.ts

import { BaseEntity, Column, Entity, PrimaryGeneratedColumn } from 'typeorm';

@Entity({name: 'UserTable'})
class UserEntity extends BaseEntity {

  @PrimaryGeneratedColumn()
  id: number;

  @Column()
  name: string;

  @Column()
  email: string;
}

export default UserEntity;

Note that this entity is given the name ‘UserTable’ which is optional but in case of migration it becomes somewhat useful. We will get to know the reason shortly.

Now create a migration file for this user entity. Migration file can be created using a command-line interface with the following command:


typeorm migration:create -n CreateUserTable

This will create a migration file with the timestamp as a substring in the name of this file.

Here, ‘CreateUserTable‘ will be the name of your migration file created by the TypeORM environment. Now we will create a folder ‘migrations’ inside the ‘db’ folder and place the migration file inside it if it is not done already.

Now create a separate file that will be used as a migration utility to decide the schema of the database. Thus, we can name this file as migrationUtil.ts

Inside this migration util file, create functions to get various types of columns namely-varchar, integer etc.

We will be creating two functions for illustration, namely ‘getIDColumn‘ and ‘getVarCharColumn‘.

#src/util/migrationUtil.ts

import { TableColumnOptions } from 'typeorm/schema-builder/options/TableColumnOptions';

class MigrationUtil {

  public static getIDColumn(): TableColumnOptions[] {
    const columns: TableColumnOptions[] = [];
    columns.push({
      name: 'userId',
      type: 'int',
      isPrimary: true,
      isNullable: false,
      isGenerated: true,
      generationStrategy: 'increment',
    });

    return columns;
  }

  public static getVarCharColumn({ name, length = '255', isPrimary = false, isNullable = false, isUnique = false, defaultValue = null }): TableColumnOptions {
    return {
      name,
      length,
      isPrimary,
      isNullable,
      isUnique,
      default: `'${defaultValue}'`,
      type: 'varchar',
    };
  }
}

export default MigrationUtil;

Here, ‘TableColumnOptions’ is a type provided by typeorm out of the box.

The code for this file is pretty straight whenever each of these functions are called, they create a separate column in your entity table.

Now, back to the ‘CreateUserTable’ migration file, the file should look like this:

#src/db/migrations/1578306918674-CreateUserTable.ts

import { MigrationInterface, QueryRunner, Table } from 'typeorm';

export class CreateUserTable1578306918674 implements MigrationInterface {

    public async up(queryRunner: QueryRunner): Promise<any> {
        
    }

    public async down(queryRunner: QueryRunner): Promise<any> {
        
    }

}

Now, add a table to this migration file using our migration utility file as:

#src/db/migrations/1578306918674-CreateUserTable.ts

....
private static readonly table = new Table({
        name: 'UserTable',
        columns: [
          ...MigrationUtil.getIDColumn(),
          MigrationUtil.getVarCharColumn({name: 'name'}),
          MigrationUtil.getVarCharColumn({name: 'email'}),
        ],
    });

....

Note that the name of this table is given same as the userEntity so as to improve entity-table mapping for developers. Also, finish up the code for async ‘up’ and ‘down’ methods using QueryRunner.

The idea is to create three columns in the user table – ‘userId’, ‘name’ and ’email’.

Thus, in the end, the migration file will be looking something like this:

#src/db/migrations/1578306918674-CreateUserTable.ts

import { MigrationInterface, QueryRunner, Table } from 'typeorm';
import MigrationUtil from '../../util/migrationUtil';

export class CreateUserTable1578306918674 implements MigrationInterface {

    private static readonly table = new Table({
        name: 'UserTable',
        columns: [
          ...MigrationUtil.getIDColumn(),
          MigrationUtil.getVarCharColumn({name: 'name'}),
          MigrationUtil.getVarCharColumn({name: 'email'}),
        ],
    });

    public async up(queryRunner: QueryRunner): Promise<any> {
        await queryRunner.createTable(CreateUserTable1578306918674.table);
    }

    public async down(queryRunner: QueryRunner): Promise<any> {
        await queryRunner.dropTable(CreateUserTable1578306918674.table);
    }

}

Now, create your environment files containing environment variables. We will be creating two .env files, namely- development.env and test.env.

The environment variables for development.env will be:

#env/development.env

TYPEORM_CONNECTION = postgres
TYPEORM_HOST = 127.0.0.1
TYPEORM_USERNAME = root
TYPEORM_PASSWORD = root
TYPEORM_DATABASE = dotenv
TYPEORM_PORT = 5432
TYPEORM_ENTITIES = db/entities/*.entity{.ts,.js}
TYPEORM_MIGRATIONS = db/migrations/*{.ts,.js}
TYPEORM_MIGRATIONS_RUN = src/db/migrations
TYPEORM_MIGRATIONS_DIR = src/db/migrations
HTTP_PORT = 3001

And the environment variables for test.env will be:

#env/test.env

TYPEORM_CONNECTION = postgres
TYPEORM_HOST = 127.0.0.1
TYPEORM_USERNAME = root
TYPEORM_PASSWORD = root
TYPEORM_DATABASE = dotenv-test
TYPEORM_PORT = 5432
TYPEORM_ENTITIES = db/entities/*.entity{.ts,.js}
TYPEORM_MIGRATIONS = db/migrations/*{.ts,.js}
TYPEORM_MIGRATIONS_RUN = src/db/migrations
TYPEORM_ENTITIES_DIR = src/db/entities
HTTP_PORT = 3001

Now, create a TypeORM config file for the connection setup.

We will place this file in the ‘config‘ folder under ‘src‘ folder of the project.

#src/config/database.config.ts

import * as path from 'path';

const baseDir = path.join(__dirname, '../');
const entitiesPath = `${baseDir}${process.env.TYPEORM_ENTITIES}`;
const migrationPath = `${baseDir}${process.env.TYPEORM_MIGRATIONS}`;

export default {
  type: process.env.TYPEORM_CONNECTION,
  host: process.env.TYPEORM_HOST,
  username: process.env.TYPEORM_USERNAME,
  password: process.env.TYPEORM_PASSWORD,
  database: process.env.TYPEORM_DATABASE,
  port: Number.parseInt(process.env.TYPEORM_PORT, 10),
  entities: [entitiesPath],
  migrations: [migrationPath],
  migrationsRun: process.env.TYPEORM_MIGRATIONS_RUN === 'true',
  seeds: [`src/db/seeds/*.seed.ts`],
  cli: {
    migrationsDir: 'src/db/migrations',
    entitiesDir: 'src/db/entities',
  },
};

Here, process.env will contain all our environment variables.

Note that the environment will be specified by us during command execution and thus, anyone of the files ‘development.env’ or ‘test.env’ will be taken as environment variables supplying file.

In the same folder, create another configuration file for dotenv and we will name it as ‘dotenv-options.ts’.

#src/config/dotenv-options.ts

import * as path from 'path';

const env = process.env.NODE_ENV || 'development';
const p = path.join(process.cwd(), `env/${env}.env`);
console.log(`Loading environment from ${p}`);
const dotEnvOptions = {
  path: p,
};

export { dotEnvOptions };

The code for this file is pretty straight.

Note that the line of code containing console.log call will let us know which environment is taken by the nest while executing commands and the same file is being provided as dotenv options below it.

Now, to successfully integrate dotenv with nest, it is recommended by official nest docs to create a config service along with a config module.

Thus, create a ‘services’ folder and inside that folder- create a ‘config.service.ts’ file.

#src/Services/config.service.ts

import * as dotenv from 'dotenv';
import * as fs from 'fs';
import * as Joi from '@hapi/joi';
import { Injectable } from '@nestjs/common';
import IEnvConfigInterface from '../interfaces/env-config.interface';
import { TypeOrmModuleOptions } from '@nestjs/typeorm';
import * as path from 'path';

@Injectable()
class ConfigService {
  private readonly envConfig: IEnvConfigInterface;

  constructor(filePath: string) {
    const config = dotenv.parse(fs.readFileSync(filePath));
    this.envConfig = this.validateInput(config);
  }

  public getTypeORMConfig(): TypeOrmModuleOptions {
    const baseDir = path.join(__dirname, '../');
    const entitiesPath = `${baseDir}${this.envConfig.TYPEORM_ENTITIES}`;
    const migrationPath = `${baseDir}${this.envConfig.TYPEORM_MIGRATIONS}`;
    const type: any = this.envConfig.TYPEORM_CONNECTION;
    return {
      type,
      host: this.envConfig.TYPEORM_HOST,
      username: this.envConfig.TYPEORM_USERNAME,
      password: this.envConfig.TYPEORM_PASSWORD,
      database: this.envConfig.TYPEORM_DATABASE,
      port: Number.parseInt(this.envConfig.TYPEORM_PORT, 10),
      logging: false,
      entities: [entitiesPath],
      migrations: [migrationPath],
      migrationsRun: this.envConfig.TYPEORM_MIGRATIONS_RUN === 'true',
      cli: {
        migrationsDir: 'src/db/migrations',
        entitiesDir: 'src/db/entities',
      },
    };
  }

  /*
	  Ensures all needed variables are set, and returns the validated JavaScript object
	  including the applied default values.
  */
  private validateInput(envConfig: IEnvConfigInterface): IEnvConfigInterface {
    const envVarsSchema: Joi.ObjectSchema = Joi.object({
      NODE_ENV: Joi.string()
        .valid('development', 'test')
        .default('development'),
      HTTP_PORT: Joi.number().required(),
    }).unknown(true);

    const { error, value: validatedEnvConfig } = envVarsSchema.validate(
      envConfig,
    );
    if (error) {
      throw new Error(`Config validation error: ${error.message}`);
    }
    return validatedEnvConfig;
  }
}

export default ConfigService;

Here, ‘IEnvConfigInterface‘ is an interface provided explicitly by us to improve the understandability of code.


export default interface IEnvConfigInterface {
  [key: string]: string;
}

The dotenv.parse will read the contents of the file containing environment variables and is made available for use. It can accept string or buffer and convert it into an object of key-value pairs.

This object is then validated by using Joi schema object which is a library provided by hapi. Under this schema, we have specified that the environment (whether test or development) will be grabbed as the NODE_ENV key in the command line.

Also, if no environment is specified, then set the environment to ‘development’. Thus, our envConfig variable is now initialized with this validated object.

Now, create a configModule and import it to app module.

#src/modules/config.module.ts

import { Global, Module } from '@nestjs/common';
import ConfigService from './Services/config.service';

@Global()
@Module({
  providers: [
    {
      provide: ConfigService,
      useValue: new ConfigService(`env/${process.env.NODE_ENV || 'development'}.env`),
    },
  ],
  exports: [ConfigService],
})
export default class ConfigModule {
}

Here config service is injected into this module. But since our config service is expecting an argument through the constructor, we will use ‘useValue’ to provide this service an argument which by default is development.env file, if no environment is explicitly provided during execution of cli commands.

Now we will create another loader file that will load all the configurations for database and dotenv.

We will create this file in ‘cli’ folder under ‘src’ folder of our project and name it as ‘loader.ts’.


import * as dotenv from 'dotenv';
import { dotEnvOptions } from '../config/dotenv-options';

// Make sure dbConfig is imported only after dotenv.config

dotenv.config(dotEnvOptions);
import * as dbConfig from '../config/database.config';

module.exports = dbConfig.default;

Note that there is a comment in the code to import dbConfig only after dotenv config is imported. This is because our database configuration will depend on the environment used by nest.

Now in our package.json file under the ‘scripts’ section, we will add two key-value pairs that will be our cli command for migration.

...

"migrate:all": "ts-node ./node_modules/typeorm/cli migration:run -f src/cli/loader.ts",
"migrate:undo": "ts-node ./node_modules/typeorm/cli migration:revert -f src/cli/loader.ts"

...

Note that this command will directly execute our loader file.

And, that’s it!

We have successfully integrated dotenv with NestJS and TypeORM.

To test this, start your database server, and then run the following cli commands one after another:


NODE_ENV=development yarn migrate:all
NODE_ENV=test yarn migrate:all

It will console the environment currently being used by us, which can be seen below:

Frequently Asked Questions

How to integrate dotenv configuration with typeorm for cli commands

A loader.ts file needs to be created, which is configured to import typeorm related configuration from respected environment file depending upon the environment. this loader is piped to typeorm cli command by using -f option. This force typeorm to load configuration from respected environment variable. The working example is given above

Why should we use dotenv with NestJS

Dotenv helps you to configure multiple env files based on the environment, and you can configure which environments to use during runtime by NODE_ENV variable. This is must for any production level config.

How to use .env params, based on different environment

You can use dotenv to configure multiple environments variables, the working example is given above.

Can we use dotenv to load config from some remote servers.

Yes you can definietly use dotenv to load configuration from remote servers, for the cli, you have to modify the loader.ts file and for the config, you have to configure the configservice.

How useful was this post?

How useful was this post?

Click on a star to rate it!

Average rating 0 / 5. Vote count: 0

No votes so far! Be the first to rate this post.

Please do Rate Us and Share!

Related Blogs

  • author
    Adam Davidson

    Best Android Emulator for PC

    Android emulator for PC or MACs is one of the best for gamers to focus and improve their gaming skills.There are many reasons for emulating Android on your Windows PC, because with the help of emulation it is much easier to test apps on-screen or desktop than a mobile device. Android developers can debug...

  • author
    Adam Davidson

    Companies that Use Node JS in Production

    Despite of being arrived late on the scene, NodeJS is dominating the entire application development scenario with its great optimal features. This is a well-kept secret for the seamless distribution of their services that some of the top companies that use NodeJS for its web-based applications today. From concurrence to being a lightweight runtime,...

  • author
    Lucas White

    How to Use callBack With setState in React

    Today we are going to explore the callback function in setState and get to know about how we can use it.  First of all, I’d like to explain the ‘callBack’ and ‘setState’. callBack functions is a function that is passed as an argument to another function, to be “called back” at a later time....

image

About The Author

William is a CTO and a full-stack engineer with 10 years of experience. He has spent the past seven years doing web and mobile apps. He’s good at designing architecture and implementing agile development process. The technologies he’s worked with include: Node.js, Elixir, Rails, AngularJS, React, React Native, Objective-C, iOS, Java, Android. He’s also familiar with C++, Haskell, C#/.NET. He is an enthusiastic programmer and a great guy to know

Try our One-Week Risk Free Trial for Hiring a Coder

Know more Hire a Coder