V2 Credit Scoring Update

Jan 18

TL;DR: In general, the majority of borrowers will see an improvement in their credit scores while some will slightly worsen.

The RociFi credit scoring model has recently undergone a few changes in an effort to improve user experience, expand data coverage, and reduce risk to lenders. This improvement process is not dissimilar to our V2 lending protocol upgrade. The credit model enhancements are likely to have a positive impact on scores for the majority of Roci NFCS holders.

V1 credit scoring was a highly optimized machine learning model that delivered solid results given this type of lending had never existed prior — overall repayment rate of 80%. The high repayment rate coupled with RociFi as first loss capital, yielded 7% or 14% annualized to lenders who never withdrew their deposits.

As our data coverage grew, ~1B records, the data science team identified several areas of improvement; including run-time speed and risk classification. The first set of improvements, V2 credit model, are now live in parallel with RociFi’s V2 lending protocol launch.

V2 has migrated to a more sophisticated machine learning approach that better captures conditional and non-linear correlations that exist between features mined from borrower historical data across 8 different blockchains. In turn, cross validation against the new model suggests a superior ability to order risk out of sample, meaning that as the score gets worse, so does the average observed default rate.

V2’s improved calibration allows us to offer more competitive lending terms, collateral ratios, and interest rates to borrowers while ensuring lenders that loan pricing more accurately reflects risk.

Below we compare the scoring distributions for Roci Borrowers under V1 and V2

Although the majority of borrowers still remain at score 10, the overall amount is lower than in V1. The reason for so many borrowers at score 10 relates to the level of borrowing activity. Both the current and previous implementation of the model are designed to penalize borrowers with light credit history. On average, any borrower with 2 or fewer loans should expect a worse score than users with a heavier borrowing history.

As for why borrower scores improved, V2 does a better job at identifying interactions that exist between wealth factors, liquidity factors, and erraticity factors. As a result, V2 more accurately classifies borrowers in lower (better) scores based on out-of-sample observed default rates.

Under V2, we expect risk to be better ordered out of sample as compared to V1. To explain, we performed a cross validation of both versions against the current data set by performing 100 trials using different train and test sets drawn per trial on both Roci-only borrowers and the greater ecosystem. The results suggest the following risk distribution assuming flat or rising markets for WETH*

V2’s estimated default rates per score are slightly higher on average than V1, but estimated defaults increase with credit score (risk). This is to be expected given the worse the score, higher the credit risk. To manage said risk, as score increases, so does the collateral ratio, i.e. protection to lenders.

In fact, under the V2 lending protocol, only the best borrowers, scores 1–4, are offered under-collateralized terms. The remainder receive highly competitive over-collateralized terms as to build their credit history within the Roci ecosystem.

The precise ways borrowers can improve their score under V2 is nuanced given the non-linear nature of the model, i.e. it may depend on more than one of your behaviors. However, there are general themes we observe

1. Use DeFi Lending Protocols. Borrowers who take and repay more loans, without being liquidated, will generally see their scores improve.
2. Hold more liquidity across wallets covered as part of your NFCS. Generally speaking, borrowers with greater assets visible to the model can expect better scores. For example, adding multiple wallets that hold liquid assets to your NFCS bundle.
3. Have those funds be liquid. Borrowers with a lot of their liquidity tied up in Liquidity Farming or other schemes, can expect worse scores than borrowers who demonstrate their assets are readily available to be used for repayment.
4. Have consistent, but not too consistent, borrow and repay behavior. Borrowers with very erratic behavior in terms of borrow and repay sizes tend to score more poorly than borrowers who are consistent. Conversely borrowers with behavior that is almost perfectly consistent, i.e. bot, may score worse. Thus, having a small amount of variability (being human) and not being erratic are associated with better credit scores.

*Please note that these estimated default risks should not be used as the basis of a decision to deposit into RociFi pools. Lenders should do careful due diligence based on the current state of the market, and their own exposures and risk tolerances. These assumptions may not hold true if the assumption that the distribution of borrowers for which use RociFi’s protocol are consistent with the distribution of borrowers used to fit the model does not hold. In addition these estimated default risks may not hold if the assumptions loans are overcollateralized and subject to auto liquidation are not met. These assumptions also not hold if the assumption the state of the market is flat or rising is not met, in particular for WETH collateral given falling markets tend to drive default rates higher than model implied default rates.*