Approach to a microservices-based architecture bank application

Published on 14 min read
View on GitHub

In microservices-based architecture, applications are built as a suite of services that are managed independently. This translates into an advantage over monolithic applications in the sense that they are easier to maintain and scale.

I am introducing in this post my own approach to a bank application, with a microservices-based architecture and using the MERN stack, which can manage accounts, customers, and loans, but also make predictions of potential loan defaults and store data on-chain.

In this article

Overview

The purpose of this dummy bank application is to develop a user interface that allows employees to create, update, and delete accounts, customers and loans, store loans on-chain and make predictions about potential loan defaults.

Therefore, this application is made up of multiple services that may or may not communicate with each other to successfully perform an action.

To build the application, I have developed a monorepository using Turborepo and Pnpm workspaces. Each microservice has been dockerized so that each service lives in its own container.

The folders packages and services gather all the workspaces.

bank-microservices
├── packages
│   ├── bank-utils
│   │   └── ...
│   ├── eslint-config-custom
│   │   └── ...
│   ├── eslint-config-custom-next
│   │   └── ...
│   ├── eslint-config-custom-subgraph
│   │   └── ...
│   ├── logger
│   │   └── ...
│   ├── server
│   │   └── ...
│   └── ts-config
│       └── ...
├── services
│   ├── accounts
│   │   └── ...
│   ├── client
│   │   └── ...
│   ├── customers
│   │   └── ...
│   ├── graph-node
│   │   └── ...
│   ├── loans
│   │   └── ...
│   ├── loans-model
│   │   └── ...
│   └── requests
│       └── ...
├── docker-compose.yaml
├── package.json
└── ...

In packages we find npm packages that contain code and configuration files that are used by other packages or services, and services gathers the set of microservices that make up the bulk of the application.

Services accounts, customers, loans , loans-model and requests start a web server application, client starts a Next.js application and data derived from a TheGraph node are stored ingraph-node.

In addition, two services, customers and loans, include a Hardhat and Foundry environment for developing, testing, and deploying contracts.

Lastly, the back-end and front-end applications are implemented based on a hexagonal architecture.

Microservices

The microservices included in this application are a collection of granular services that mostly work independently although some of them have some dependency on other services.

For example, services customers and loans will start working satisfactorily when the TheGraph node is active in order to be able to deploy their own subgraphs and thus client can consume the data.

Additionally, all services running an Express.js server depend on access to different MongoDB databases that reside in separate Docker containers.

For the rest, only the client will be the one that communicates with the other services to be able to complete their functions.

The services that make up this application and its main characteristics are the following:

  • accounts: this microservice is in charge of all the actions that are related to the accounts in the bank: create, delete, update and get accounts, deposit to accounts and withdraw from accounts.

  • client: it is a Next.js application that a user can use to execute the relevant actions with which to interact with the rest of the microservices. You can find more information in user interface.

  • customers: this service implements all the actions that have to do with bank customers: create, delete, update and get customers.

  • graph node: data from the PostgresSQL database and from the IPFS network used by TheGraph node will be written in this workspace for its correct operation.

The node configuration along with the configuration of the rest of the Docker images in this application are in the file /docker-compose.yaml.

  • loans: this microservice takes care of all the actions that are related to loans: create, delete, update and get loans.

  • loans model: this service implements a pipeline formed by transformer objects and a neural network, which make up a workflow to classify, binarily, loans and thus predict whether or not they may incur in default. You can find more information in classification model.

  • requests: this service exists to represent hypothetical loan requests from customers, from which to obtain information to interact with other services.

In microservices, therefore, you can find implementations of Express.js web services with access to independent MongoDB databases, as well as Hardhat and Foundry environments for developing smart contracts, subgraphs for indexing data from the events in these contracts, a Flask web service to interact with a binary classification model, and a Next.js application to shape a user interface.

The following sections introduce all of these features by referring to one of the services to attach code as an example.

Back end

Versioned APIs have been developed for four of the microservices implemented in the application, in such a way that persistent data can be created, read, updated and deleted through an Express.js server.

The architecture used for all of them is hexagonal. Next, the main features of the API developed for the loans microservice will be introduced.

File structure

The basis on which the files are structured is as follows:

bank-microservices
├── services
│   ├── loans
│   │   ├── api
│   │   │   └── v1
│   │   │       ├── controllers
│   │   │       │   └── ...
│   │   │       ├── core
│   │   │       │   ├── interactors
│   │   │       │   │   └── ...
│   │   │       │   └── repositories
│   │   │       │       └── ...
│   │   │       ├── data-sources
│   │   │       │   └── ...
│   │   │       ├── index.ts
│   │   │       └── ...
│   │   └── ...
│   └── ...
└── ...

The server will be instantiated in the file index.ts passing it an object with information about routes, an instance of a WinstonLogger and optionally other configuration data.

Both the implementation of the logger as of the server are in /packages.

services/loans/api/v1/index.ts
const logger: IWinstonLogger = new WinstonLogger();
 
const deleteByIdPath = join(deletePath, ':identifier');
const getByIdPath = join(getPath, ':identifier');
 
const routesData: RouteData[] = [
  // Valid requests
  { method: createMethod, path: createPath, handler: createLoanController },
  { method: deleteByIdentifierMethod, path: deleteByIdPath, handler: deleteLoanByIdentifierController },
  { method: getAllMethod, path: getAllPath, handler: getAllLoansController },
  { method: getByIdentifierMethod, path: getByIdPath, handler: getLoanByIdController },
  { method: updateMethod, path: updatePath, handler: updateLoanController },
  // Invalid methods
  { method: 'all', path: createPath, handler: handleInvalidMethodController, isSync: true },
  { method: 'all', path: deleteByIdPath, handler: handleInvalidMethodController, isSync: true },
  { method: 'all', path: getAllPath, handler: handleInvalidMethodController, isSync: true },
  { method: 'all', path: getByIdPath, handler: handleInvalidMethodController, isSync: true },
  { method: 'all', path: updatePath, handler: handleInvalidMethodController, isSync: true },
  // Unrecognized URLs
  { method: 'all', path: '*', handler: handleNotFoundController, isSync: true }
];
 
// const config: Config = {
//   cache: { duration: '5 minutes', onlySuccesses: true },
//   cors: {
//     origin: `http://localhost:${LoansConfigV1.CLIENT_PORT ?? 3000}`,
//     optionsSuccessStatus: 200
//   }
// };
 
async function main(): Promise<void> {
  new ExpressApi(logger, routesData).start(LoansConfigV1.PORT ?? '3003');
}
 
main().catch((error) => {
  logger.error(error);
  process.exit(0);
});

When running this script, the server will be available on the port that is passed to it as an argument and that will be assigned in an environment variable file.

In the file scheme that we saw above we see that the API is made up of three folders:

  • controllers: they are in charge of defining the operations desired by the API of the application. For example, when doing a fetch to the loans server on the route /v1/create with the POST method, the server will assign the following controller to take care of performing the desired actions:

    services/loans/api/v1/controllers/create-loan.controller.ts
    export const createLoanController = async ({ body }: Request, response: Response): Promise<void> => {
  const { loan } = body;
 
  // if body is not an instance of Loan
  if (!isInstanceOfLoan(loan)) {
    response.status(StatusCodes.BAD_REQUEST).json({ success: false, error: true, message: loanBadRequestMessage, data: null });
  }
  // If request is valid, call the interactor
  else {
    const { success, ...rest } = await createLoanWithDep(loan);
    // If the request succeeded
    if (success) {
      response.status(StatusCodes.OK).json({ success, ...rest });
    }
    // If the request did not succeed
    else {
      response.status(StatusCodes.INTERNAL_SERVER_ERROR).json({ success, ...rest });
    }
  }
};

    As we can see, in case the body of the request is an instance of the entity Loan, that is, if the request is valid, an interactor will be executed which has already been injected with the loans repository as an abstraction of a data source class, where that repository is implemented.

    Entities, as well as other tools that are shared between several services are implemented in the package bank-utils. For example, the entity Loan imported from there can be seen next:

    packages/bank-utils/
    export interface Loan {
  creditPolicy: boolean;
  customerId: number;
  delinquencies2Yrs: number;
  daysWithCreditLine: number;
  identifier: number;
  installment: number;
  intRate: number;
  isDefault: boolean;
  purpose: string;
  revolBal: number;
  revolUtil: number;
}

    Below we have a diagram in which the code flow is indicated when executing the controller createLoanController:

    Therefore, the result will be obtaining an object of type InteractorResponse, which will give us information about the status of the request as well as messages and data.

    In this way, the timely response from the server will be given in each case, as we can see in the implementation of createLoanController in lines 6, 13 and 17.

  • core: the folder core collects the business logic of each service. It is made up of repositories and interactors. As we saw earlier, the controller will invoke at least one use case, or interactor, that interacts with a repository to get an entity.

    • repositories: they are interfaces that define all the methods that are necessary to deal with all the use cases of each service.

      These methods are implemented in a data source class outside of the core.

      For example, we can see part of the LoanRepository below:

      services/loans/api/v1/core/repositories/loan.repository.ts
      interface LoanRepository {
  /**
  * Connect to MongoDB using a url.
  */
  connect: () => Promise<void>;
 
  /**
  * Close the client and its underlying connections.
  */
  close: () => Promise<void>;
 
  /**
  * Create a loan.
  * @param loan `Loan` entity object.
  */
  create: (loan: Loan) => Promise<void>;
 
  // ...
}
 
export default LoanRepository;

    • interactors: the use cases of the corresponding service are implemented in the interactors.

      Controllers use interactors that already have dependencies injected to keep business logic independent of the framework.

      For example, we previously saw in controllers that the interactor createLoanWithDep was used, which already has its dependency injected as we can see below:

      services/loans/api/v1/core/interactors/index.ts
      const loanRepository = new LoanDataSource();
 
const createLoanWithDep = createLoan(loanRepository);
const deleteLoanByIdWithDep = deleteLoan(loanRepository);
const getLoanByIdWithDep = getLoanById(loanRepository);
const getAllLoansWithDep = getAllLoans(loanRepository);
const handleInvalidMethodWithDep = handleInvalidMethod(loanRepository);
const handleNotFoundWithDep = handleNotFound(loanRepository);
const updateLoanWithDep = updateLoan(loanRepository);
 
export {
  createLoanWithDep,
  deleteLoanByIdWithDep,
  getLoanByIdWithDep,
  getAllLoansWithDep,
  handleInvalidMethodWithDep,
  handleNotFoundWithDep,
  updateLoanWithDep
};

      The implementation of the interactor createLoan can be seen below:

      services/loans/api/v1/core/interactors/create-loan.interactor.ts
      const createLoan =
  (repository: LoanRepository) =>
  async (loan: Loan): Promise<InteractorResponse> => {
    try {
      await repository.connect();
      await repository.create(loan);
      return { success: true, error: false, message: 'A loan has been succesfully created.', data: null };
    } catch (error) {
      repository.log('error', error);
      if (error instanceof Error) {
        const { message } = error;
        return { success: false, error: true, message, data: null };
      } else {
        return { success: false, error: true, message: errorMessage, data: null };
      }
    } finally {
      try {
        await repository.close();
      } catch (error) {
        if (error instanceof Error) {
          repository.log('warn', error.stack ?? error.message);
        } else {
          repository.log('warn', clientClosingErrorMessage);
        }
      }
    }
  };
 
export default createLoan;

      As we can see, this interactor expects an entity LoanRepository which is used to call four of its methods (log, connect, create and close), but the interactor does not depend on its implementation.

  • data sources: as we already said, the methods that are described in the repository are implemented in a class in the folder data-sources.

    We can see below the implementation of the four methods mentioned above and that are used in the interactor createLoan:

    services/loans/api/v1/data-sources/loan.data-source.ts
    class LoanDataSource implements LoanRepository {
  private client: MongoClient | null;
  private readonly logger: IWinstonLogger;
 
  constructor() {
    this.client = null;
    this.logger = new WinstonLogger();
  }
 
  public async connect(): Promise<void> {
    if (typeof DB_PORT === 'undefined' || typeof DB_USER === 'undefined' || typeof DB_PASSWORD === 'undefined' || typeof DB_URI === 'undefined') {
      throw new Error('Cound not read configuration data');
    }
    const URI = typeof process.env.NODE_ENV !== 'undefined' && process.env.NODE_ENV === 'development' ? `mongodb://localhost:${DB_PORT}` : DB_URI;
    this.client = new MongoClient(URI, { auth: { username: DB_USER, password: DB_PASSWORD } });
    await this.client.connect();
  }
 
  public async close(): Promise<void> {
    if (this.client !== null) {
      await this.client.close();
      this.client = null;
    }
  }
 
  public async create({ identifier, ...rest }: Loan): Promise<void> {
    const loans = this.getCollection();
    const loan = await loans.findOne<Loan>({ identifier });
    if (loan !== null) {
      throw new Error(`There is already a loan with the identifier: ${identifier}`);
    }
    await loans.insertOne({ identifier, ...rest });
  }
 
  public log(type: 'info' | 'warn' | 'error', message: any): void {
    if (type === 'info') {
      this.logger.info(message);
    } else if (type === 'warn') {
      this.logger.warn(message);
    } else {
      this.logger.error(message);
    }
  }
  // ...
}

Smart contracts

In the financial sphere, there are numerous opportunities to make use of the advantages of blockchains. The decentralisation of the data in this technology ensures the security and immutability that is required to be able to save other bureaucratic procedures.

For example, requirements imposed on providers by different regulations result in a long time for the procedures to be expedited and the funds to be provided.

In the event that these procedures could be expedited by securely sharing information between approved credit entities, credit decision time could be substantially reduced.

In this dummy application I have written two very simple contracts in which the necessary methods are implemented to store and update both loans and customers.

Next we can see the functions of the contract LoanManager:

services/loans/contracts/src/LoanManager.sol
contract LoanManager is Ownable {
    // ...
    /**
     * Add a loan.
     * @param identifier loan identifier.
     * @param customerId customer id.
     * @param purpose purpose of the loan.
     * @param intRate interest rate.
     * @param installment installment.
     * @param delinquencies2Yrs delinquencies in the last 2 years.
     * @param isDefault bool that represents whether this loan is default.
     */
    function addLoan(
        uint256 identifier,
        uint256 customerId,
        bytes32 purpose,
        bytes32 intRate,
        bytes32 installment,
        uint256 delinquencies2Yrs,
        bool isDefault
    ) public onlyOwner {
        _loanIdToLoan[identifier].isLoan = true;
        _loanIdToLoan[identifier].customerId = customerId;
        _loanIdToLoan[identifier].purpose = purpose;
        _loanIdToLoan[identifier].intRate = intRate;
        _loanIdToLoan[identifier].installment = installment;
        _loanIdToLoan[identifier].delinquencies2Yrs = delinquencies2Yrs;
        _loanIdToLoan[identifier].isDefault = isDefault;
        emit LoanAdded(identifier, customerId, purpose, intRate, installment, delinquencies2Yrs, isDefault);
    }
 
    /**
     * Update a loan.
     * @param identifier loan identifier.
     * @param purpose purpose of the loan.
     * @param intRate interest rate.
     * @param installment installment.
     * @param delinquencies2Yrs delinquencies in the last 2 years.
     * @param isDefault bool that represents whether this loan is default.
     */
    function updateLoan(
        uint256 identifier,
        bytes32 purpose,
        bytes32 intRate,
        bytes32 installment,
        uint256 delinquencies2Yrs,
        bool isDefault
    ) public onlyOwner {
        require(_loanIdToLoan[identifier].isLoan == true, 'Loan does not exist.');
        _loanIdToLoan[identifier].purpose = purpose;
        _loanIdToLoan[identifier].intRate = intRate;
        _loanIdToLoan[identifier].installment = installment;
        _loanIdToLoan[identifier].delinquencies2Yrs = delinquencies2Yrs;
        _loanIdToLoan[identifier].isDefault = isDefault;
        emit LoanUpdated(identifier, purpose, intRate, installment, delinquencies2Yrs, isDefault);
    }
 
    /**
     * Get loan data by identifier.
     * @param identifier loan identifier.
     * @return isLoan bool that represents whether this loan has been added.
     * @return customerId customer id.
     * @return purpose purpose of the loan.
     * @return intRate interest rate.
     * @return installment installment.
     * @return delinquencies2Yrs delinquencies in the last 2 years.
     * @return isDefault bool that represents whether this loan is default.
     */
    function getLoanById(
        uint256 identifier
    )
        external
        view
        returns (
            bool isLoan,
            uint256 customerId,
            bytes32 purpose,
            bytes32 intRate,
            bytes32 installment,
            uint256 delinquencies2Yrs,
            bool isDefault
        )
    {
        Loan memory loan = _loanIdToLoan[identifier];
        return (
            loan.isLoan,
            loan.customerId,
            loan.purpose,
            loan.intRate,
            loan.installment,
            loan.delinquencies2Yrs,
            loan.isDefault
        );
    }
}

Testing the contracts

As already mentioned, there is also an environment of Foundry to be able to test the contracts in a very intuitive and fast way using the language Solidity.

Below we can see part of the test contract to test the functions in the contract LoanManager:

services/loans/contracts/test/foundry/LoanManager.t.sol
contract LoanManagerTest is Test {
    // ...
    function setUp() public {
        debtorsData = new LoanManager();
    }
 
    function testAddLoan() public {
        vm.expectEmit(true, true, true, true);
        emit LoanAdded(_loanId, _customerId, _purpose, _intRate, _installment, _delinquencies2Yrs, _isDefault);
        debtorsData.addLoan(_loanId, _customerId, _purpose, _intRate, _installment, _delinquencies2Yrs, _isDefault);
        (bool isLoan, , , , , , ) = debtorsData.getLoanById(_loanId);
        assertEq(isLoan, true);
    }
 
    function test_RevertedWhen_CallerIsNotOwner_AddLoan() public {
        vm.expectRevert('Ownable: caller is not the owner');
        vm.prank(address(0));
        debtorsData.addLoan(_loanId, _customerId, _purpose, _intRate, _installment, _delinquencies2Yrs, _isDefault);
    }
 
    function testUpdateLoan() public {
        debtorsData.addLoan(_loanId, _customerId, _purpose, _intRate, _installment, _delinquencies2Yrs, false);
        vm.expectEmit(true, true, true, true);
        emit LoanUpdated(_loanId, _purpose, _intRate, _installment, _delinquencies2Yrs, true);
        debtorsData.updateLoan(_loanId, _purpose, _intRate, _installment, _delinquencies2Yrs, true);
        (, , , , , , bool isDefault) = debtorsData.getLoanById(_loanId);
        assertEq(isDefault, true);
    }
 
    function test_RevertedWhen_LoanDoesNotExist_UpdateLoan() public {
        vm.expectRevert('Loan does not exist.');
        debtorsData.updateLoan(_loanId, _purpose, _intRate, _installment, _delinquencies2Yrs, _isDefault);
    }
 
    function test_RevertedWhen_CallerIsNotOwner_UpdateLoan() public {
        vm.expectRevert('Ownable: caller is not the owner');
        vm.prank(address(0));
        debtorsData.updateLoan(_loanId, _purpose, _intRate, _installment, _delinquencies2Yrs, _isDefault);
    }
 
    function testGetLoanById() public {
        (bool isLoanWithDefaultValues, , , , , , ) = debtorsData.getLoanById(_loanId);
        assertEq(isLoanWithDefaultValues, false);
        debtorsData.addLoan(_loanId, _customerId, _purpose, _intRate, _installment, _delinquencies2Yrs, _isDefault);
        (bool isLoanAfterAddition, , , , , , ) = debtorsData.getLoanById(_loanId);
        assertEq(isLoanAfterAddition, true);
    }
}

Subgraphs

Microservices customers and loans implement subgraphs using TheGraph, which can be deployed to a TheGraph node and thus a client can consume data from it efficiently.

Since the data to be used cannot be public and given the nature of the application, it might be more appropriate to deploy the contracts to a private blockchain than to encrypt the data.

Anyway, I have developed subgraphs using TheGraph -which does not support every single blockchain and network- due to its ease of use and efficiency from the perspective of the application consuming the data.

To create these subgraphs there are three fundamental parts: the manifest, the GraphQL schema and the mappings.

Subgraph manifest

The subgraph manifest specifies which contracts the subgraph indexes, which events to react to, and how event data are mapped to entities that are stored and queried by the TheGraph node.

Below is the manifest for the loans microservice:

services/loans/subgraph/templates/subgraph.template.yaml
specVersion: 0.0.4
description: Bank Microservices Loans
repository: https://github.com/albertobas/bank-microservices/services/loans
schema:
    file: ./schema.graphql
features:
    - ipfsOnEthereumContracts
dataSources:
    - name: LoanManagerDataSource
      kind: ethereum/contract
      network: {{network}}
      source:
          address: "{{LoanManagerAddress}}"
          abi: LoanManager
          startBlock: {{LoanManagerStartBlock}}
      mapping:
          kind: ethereum/events
          apiVersion: 0.0.6
          language: wasm/assemblyscript
          file: ./src/mappings/LoanManager.ts
          entities:
              - Account
              - LoanManagerContract
              - Loan
              - LoanAddition
              - LoanUpdate
              - OwnershipTransferred
              - Transaction
          abis:
              - name: LoanManager
                file: ../contracts/deployments/{{network}}/LoanManager.json
          eventHandlers:
              - event: LoanAdded(indexed uint256,indexed uint256,bytes32,bytes32,bytes32,uint256,bool)
                handler: handleLoanAdded
              - event: LoanUpdated(indexed uint256,bytes32,bytes32,bytes32,uint256,bool)
                handler: handleLoanUpdated
              - event: OwnershipTransferred(indexed address,indexed address)
                handler: handleOwnershipTransferred

The keys that appear between braces will be replaced when generating the manifest by values ​​obtained from the contract deployment.

GraphQL schema

Considering the simplicity of the contracts whose events are to be indexed, a relatively dense and interlaced schema has been developed.

To do this, a distinction is made between the creation of entities for:

  • high-level concepts: concepts such as the entity Loan or LoanManagerContract.

    services/loans/subgraph/schema.graphql
    type LoanManagerContract @entity {
  id: Bytes!
  asAccount: Account!
  owner: Account
  loans: [Loan!]! @derivedFrom(field: "contract")
  loanAdditions: [LoanAddition!]! @derivedFrom(field: "contract")
  loanUpdates: [LoanUpdate!]! @derivedFrom(field: "contract")
}

  • low-level concepts: events that emit the contracts.

    services/loans/subgraph/schema.graphql
    type LoanAddition implements IEvent @entity(immutable: true) {
  id: ID!
  contract: LoanManagerContract!
  emitter: Account!
  loan: Loan!
  timestamp: BigInt!
  transaction: Transaction!
}

Mappings

Through the functions defined as mappings in the manifest, TheGraph assigns data that keeps obtaining from the events of the contract to entities defined in the schema.

For example, the following function is responsible for writing several entities with the data processed in the store of the TheGraph node after the contract emits the event LoanAdded.

services/loans/src/mappings/LoanManager.ts
export function handleLoanAdded(event: LoanAdded): void {
  let loan = ensureLoan(event.params.identifier, event.address, event.block.timestamp);
  let addition = registerLoanAddition(event, loan);
  loan.addition = addition.id;
  loan.customer = event.params.customerId;
  loan.installment = event.params.installment.toString();
  loan.intRate = event.params.intRate.toString();
  loan.isDefault = event.params.isDefault;
  loan.purpose = event.params.purpose.toString();
  loan.delinquencies2Yrs = event.params.delinquencies2Yrs;
  loan.save();
}

Classification model

The microservice loans-model consists of a Flask web service which gives access to a classification model.

We can predict whether a loan may default using this binary classification model after a supervised training using a database of loans from LendingClub.

This is a highly unbalanced data set, therefore the predictions of the minority class have considerably less accuracy than those of the majority class.

No replacement technique has been considered since the only object of the exercise is to be able to make predictions from the user interface.

A Scikit-learn pipeline has been used that integrates the following:

  • a preprocessing stage with a qualitative variable encoder and a transformer object for variable scaling.

  • the Skorch wrapper NeuralNetBinaryClassifier for a neural network model implemented in PyTorch.

The neural net Model and the transformer object Encoder can be found in /services/loans-model/utils. The model used is next:

services/loans-model/utils/model.py
class Model(nn.Module):
   def __init__(self, num_units, input_size, dropout=0):
       super(Model, self).__init__()
       self.dense0 = nn.Linear(input_size, num_units)
       self.dropout0 = nn.Dropout(dropout)
       self.dense1 = nn.Linear(num_units, num_units)
       self.dropout1 = nn.Dropout(dropout)
       self.dense2 = nn.Linear(num_units, num_units)
       self.dropout2 = nn.Dropout(dropout)
       self.output = nn.Linear(num_units, 1)
 
   def forward(self, X):
       X = F.relu(self.dense0(X))
       X = self.dropout0(X)
       X = F.relu(self.dense1(X))
       X = self.dropout1(X)
       X = F.relu(self.dense2(X))
       X = self.dropout2(X)
       X = self.output(X)
       X = X.squeeze(-1)
       return X

We can see the script for the training of this pipeline and its serialization for later deployment below.

services/loans-model/dump_model.py
data = fetch_openml(name="Lending-Club-Loan-Data", version=1,
                    as_frame=True, data_home='data', parser='auto')
frame = data.frame.copy()
y = frame['not.fully.paid'].astype(np.float32)
frame.drop('not.fully.paid', axis=1, inplace=True)
 
 
qualitative = ['purpose']
categories = list(itertools.chain.from_iterable((var + '_' + str(value)
                                                 for value in np.unique(frame[var].dropna()))
                                                for var in qualitative))
 
classifier = NeuralNetBinaryClassifier(
    Model(num_units=64, input_size=frame.shape[1]),
    criterion=BCEWithLogitsLoss,
    optimizer=Adam,
    batch_size=32,
    iterator_train__shuffle=True,
    lr=0.01,
    max_epochs=200)
 
pipe = Pipeline([
    ('encoder', Encoder(categories, qualitative)),
    ('scale', MinMaxScaler()),
    ('classifier', classifier),
])
 
pipe.fit(frame, y)
 
dump(pipe.best_estimator_, 'pipeline.joblib')

I have used Skorch since it offers the advantage of wrappping a model implemented with PyTorch and use it with Scikit-learn's GridSearchCV.

In this way, a hyperparameter optimization of both, model and training, evaluated by cross-validation is performed using three folds of the data set.

Here is a snippet from the Jupyter notebook loans_default.ipynb showing this implementation:

services/loans-model/loans_default.ipynb
params = {
    'classifier__lr': [0.01, 0.005, 0.001],
    'classifier__module__num_units': [32, 64, 90],
    'classifier__batch_size': [32, 64, 128]
}
 
grid_search = GridSearchCV(pipe, params, refit=True,
                           cv=3, scoring='accuracy', verbose=0)
 
grid_result = grid_search.fit(frame, y)
Best mean test score: 0.838, Best std test score: 0.003, Best params: {'classifier__batch_size': 32, 'classifier__lr': 0.01, 'classifier__module__num_units': 32}

Finally, we can see below the code used to execute the Flask app, along with the implementation of the function predict for the route /predict, which is the only route:

services/loans-model/app.py
model = load('pipeline.joblib')
app = Flask(__name__)
cors = CORS(app, resources={
            r"/predict": {"origins": f"http://localhost:{CLIENT_PORT}"}})
 
 
@app.route('/predict', methods=['POST'])
def predict():
    new_obsevation = request.get_json(force=True)
    new_observation_frame = pd.DataFrame([new_obsevation.values()], columns=['credit.policy', 'purpose', 'int.rate', 'installment', 'log.annual.inc',
                                                                             'dti', 'fico', 'days.with.cr.line', 'revol.bal', 'revol.util', 'inq.last.6mths', 'delinq.2yrs', 'pub.rec'])
    prediction = model.predict(new_observation_frame)
    output = prediction[0]
    response = jsonify(str(output))
    return response
 
 
if __name__ == '__main__':
    app.run(port=APP_PORT, debug=True)

User interface

A simple web application has been developed with Next.js 13 using the new feature App Router.

It is an application with five routes and four sections, in which some forms and buttons are used to be able to interact with the servers, as well as with the smart contracts.

For example, in every section there is a card for each use case that is implemented in its respective microservice.

Below we can see the functional component in React.js for the creation of a loan:

services/client/src/app/loans/_components/create-loan-card.tsx
export function CreateLoanCard(): JSX.Element {
  const [message, setMessage] = useState<string | null>(null);
 
  const {
    register,
    handleSubmit,
    formState: { errors },
    reset,
  } = useForm({
    resolver: yupResolver(loanSchema),
  });
 
  const onSubmit = handleSubmit(async ({ creditPolicy, isDefault, ...rest }) => {
    const response = await createLoanWithDep({
      creditPolicy: Boolean(creditPolicy),
      isDefault: Boolean(isDefault),
      ...rest,
    } as Loan);
    console.log('RESPONSE:', response);
    setMessage(response.message);
    reset();
  });
 
  return (
    <Card>
      <h2>Create loan</h2>
      <LoanForm onSubmit={onSubmit} register={register} errors={errors} />
      {message !== null && <p>{message}</p>}
    </Card>
  );
}

This is a form using React Hook Form with which the necessary data are obtained to be able to form an entity Loan and pass it to the interactor createLoanWithDep when pressing Submit.

Below we can see the flow chart of this component:

Once again, interactors call a method of a repository to get an entity, the repository in turn is injected into the interactor as an abstraction of a data source class.

File structure

As we can see in the following diagram, this Next.js application consists of the app folder that gathers all the necessary components in the required paths, plus a features folder.

bank-microservices
├── services
│   ├── client
│   │   ├── src
│   │   │   ├── app
│   │   │   │   ├── _components
│   │   │   │   │   └── ...
│   │   │   │   ├── loans
│   │   │   │   │   ├── _components
│   │   │   │   │   ├── page.tsx
│   │   │   │   │   │   └── ...
│   │   │   │   │   └── ...
│   │   │   │   ├── layout.tsx
│   │   │   │   ├── page.tsx
│   │   │   │   └── ...
│   │   │   ├── features
│   │   │   │   ├── loans
│   │   │   │   │   ├── core
│   │   │   │   │   │   ├── interactors
│   │   │   │   │   │   │   └── ...
│   │   │   │   │   │   └── repositories
│   │   │   │   │   │       └── ...
│   │   │   │   │   └── data-sources
│   │   │   │   │       └── ...
│   │   │   │   └── ...
│   │   │   ├── shared
│   │   │   │   └── ...
│   │   │   └── ...
│   │   └── ...
│   └── ...
└── ...

Let's see what these two folders consist of next.

App

The use of the App Router allows us to make use of React Server Components and thus optimize the performance of the application.

In the case of needing to add interactivity on client-side, it is only necessary to place the directive 'use client' at the top of a file.

It is also convenient to warn about the use of conventions for the names of the files. So, page and layout are repeated successively in each route.

A layout is shared across multiple pages, and each page is unique to each route.

In this way, when navigating to the route /loans we will have, among others, the card of the form that we saw above for the creation of a loan:

services/client/src/app/loans/page.tsx
export default function Page(): JSX.Element {
  return (
    <div className={styles.container}>
      <CardList>
        <CreateLoanCard />
        <UpdateLoanCard />
        <DeleteLoanCard />
        <GetAllLoansCard />
        <GetLoanCard />
        <RequestLoanDefaultCard />
        <SaveLoanOnChainCard />
      </CardList>
    </div>
  );
}

Features

The business logics and data sources for each section (accounts, customers, loans and requests) are implemented in this folder.

The file structure is similar to the one used in the server APIs:

  • core: in this folder, as we already said, we will find the business logic of each section:

    • interactors: the interactors are the result of the implementation of all the use cases of each section. They are designed to accept a repository as an argument, and return a function with this repository already injected.

      For example, below we can see the interactor createLoan:

      services/client/src/features/loans/core/interactors/create-loan.interactor.ts
      const createLoan =
  (repository: LoanRepository) =>
  async (loan: Loan): Promise<InteractorResponse> => {
    try {
      const response = await repository.create(loan);
      return response;
    } catch (error) {
      if (error instanceof Error) {
        const { message } = error;
        return { success: false, error: true, message, data: null };
      } else {
        return { success: false, error: true, message: errorMessage, data: null };
      }
    }
  };
 
export default createLoan;

    The interactor calls the method create from the repository and returns a response of the type InteractorResponse. If an error occurs, an object of the same type is generated with error messages.

  • repositories: in the repository we define the methods needed to deal with all use cases. Below we see part of the loan repository:

    services/client/src/features/loans/core/repositories/loan.repository.ts
    interface LoanRepository {
  /**
  * Create a loan.
  * @param loan `Loan` entity object.
  * @returns a Promise that resolves to an `InteractorResponse` object.
  */
  create: (loan: Loan) => Promise<InteractorResponse>;
 
  /**
  * Delete a loan by its identifier.
  * @param identifier loan identifier.
  * @returns a Promise that resolves to an `InteractorResponse` object.
  */
  deleteByIdentifier: (identifier: number) => Promise<InteractorResponse>;
 
  // ...
}
 
export default LoanRepository;

  • data sources: every use case described in the repositories is implemented in the data sources. Below we see part of the data source that corresponds to loans:

    services/client/src/features/loans/data-sources/loan.data-source.ts
    class LoansDataSource implements LoanRepository {
  public async create(loan: Loan): Promise<InteractorResponse> {
    const response = await this.fetchWrapper(createPath, createMethod, { loan });
    return response.json();
  }
 
  public async deleteByIdentifier(identifier: number): Promise<InteractorResponse> {
    const response = await this.fetchWrapper(join(deletePath, identifier.toString()), deleteByIdentifierMethod);
    return response.json();
  }
 
  // ...
 
  /**
  * Fetch wrapper.
  * @param path the path of the URL you want to conform.
  * @param method the request method.
  * @param body the body that you want to add to your request.
  * @returns a Promise that resolves to a `Response` object.
  */
  private async fetchWrapper(path: string, method: string, body?: any, isModel?: boolean): Promise<Response> {
    const validatedMethod = validateRequestMethod(method);
    if (validatedMethod === null) {
      const message = 'Invalid request method';
      throw new Error(message);
    }
    const url = new URL(path, isModel ? loanModelBase : loanBase).href;
    const bodyString = body ? JSON.stringify(body) : undefined;
    return fetchWrapper(url, validatedMethod, bodyString, isModel ? 'cors' : undefined);
  }
}
 
export default LoansDataSource;

    In line 21 we can see a private method that is used for a promise that resolves to an object of type Response.

    This is a function that is used by all use cases and that wraps the method fetch to begin the process of obtaining a resource from a server.

Finally, we can see in the following image the loans section and all its use cases:

Conclusion

In this post I have tried to introduce my approach to a simple microservice-based architecture bank application that implements the main use cases for each service.

In addition, an example has been made in which artificial intelligence and blockchain technology coexist, through the deployment of:

  • a pipeline which includes a binary classification neural network implemented with PyTorch.

  • smart contracts developed in language Solidity.

  • subgraphs using TheGraph which in practice act as an indexing layer between the contracts and the user interface.

Related posts

zk-SNARK

How to build a zero-knowledge DApp

This post offers an introduction to how to develop an application capable of generating and...

Regression

Predicting Ames housing prices: pipeline workflow

Price regression on the Ames housing prices dataset

Classification

Classifying news from 20NewsGroup

Classifying text in the 20Newsgroup dataset

Traceability

Introducing Olive Oil Trust

Introduction to a series of posts about Olive Oil Trust

DeFi

How to build a DEXs analytics application

Introduction to obtaining and representing DEXs data using TheGraph and React.js

Traceability

Introducing Olive Oil Trust: front end

Next.js application that gives support to members and customers in Olive Oil Trust, and reduces...

Ready to #buidl?

Are you interested in Web3 or the synergies between blockchain technology, artificial intelligence and zero knowledge?. Then, do not hesitate to contact me by e-mail or on my LinkedIn profile. You can also find me on GitHub.