Job Abstract

This position is an individual contributor role residing within the Bank’s Second Line of Defense Risk Management and Compliance organization. Specifically, this position supports the Model Risk Management (“MRM”) program at the Bank. The overall MRM program is designed to provide governance and control to manage financial models and associated risks that may impact the company, including financial, liquidity, market, operational, reputational, strategic, and other risks as appropriate.

Validation Analysts within MRM provide independent validation of models that the Bank relies on for making financial decisions. A robust and comprehensive model validation includes steps that independently challenge a model’s conceptual framework and methodology, data used in the model, implementation process, as well as model usage and performance. The validation process also identifies corrective actions to ensure timely remediation of model risk.

The individual in this position works with Model Owners and Developers to validate models related to Cybersecurity, adhering to OCC 2011-12 Regulatory Guidance and USB’s Model Risk Management Policy and Standards. The Analyst will be responsible for documenting and presenting the model validation findings to Model Owners/Developers, as well as interface with key stakeholders, regulators (OCC/FRB), and internal auditors to discuss these reported observations and published outcomes.

Necessary Skills

• Demonstrated ability to work independently as well as across collaborative groups.

• Strong project management skills.

• Has considerable knowledge of various machine learning algorithms and their applications, including (but not limited to): Random Forest, GBM, XGBoost, deep learning, NLP, and ensemble methods.

• Able to perform complex mathematical analyses utilizing various statistical methods or techniques including cluster analysis, KNN, outlier methods and other ensemble techniques.

• Can independently assess, test, document, and oversee the usage of advanced quantitative methods. Deliverables will include the creation of validation documentation such as: presentations, written reports, validation benchmarks models and support for identified concerns and proposed enhancements.

• Proficiency in Python, and experience with deep learning frameworks such as PyTorch, Tensorflow/Keras, MXNet.

• Can retrieve, analyze and interpret data from various data sources (internal and external).

• Can aggregate and/or create datasets from different databases (structured or non-structured) to facilitate analyses related to model development or validation.

• Able to design and perform ad hoc analyses using standard or advanced exploratory data analysis (EDA) techniques.

• Can interpret results from analyses performed and communicate outcomes to senior executive level management in written and/or verbal form.

• Can prepare and communicate analysis and outcome for presentation to audit and regulatory examiners.

• Able to read, understand, and interpret regulatory rules and guidance related to the model(s) being assessed.

• Ability to understand the business product or structure to ensure models capture the core elements and risks of interest.

• Ability to build strong relationships with stakeholders, business line partners, and colleagues across the Bank.

Preferred Skills/Experience

• Direct experience in the development and testing of machine learning models. This work could be in the form of research experience at a university or industry lab, with primary emphasis on application of complex modeling methods. Experience and authored research publications in machine learning model training, fine-tuning, reinforcement learning, or Gen AI application development/deployment are strongly preferred. 

• Advanced understanding of applicable compliance laws, regulations, financial services, and regulatory trends in the cybersecurity space.

• Prior exposure to validation of Machine Learning and NLP models.

• Prior experience working with a Financial Institution.

• Deep understanding of modeling methods for extreme imbalanced data.

• Experience using Python is required; experience using SAS, STATA, R, EVIEWS, or comparable statistical software or languages (e.g., SQL) is also desirable.

• Strong people leadership skills.

NOTE:  This role offers a hybrid/flexible schedule, which means there's an in-office expectation of 3 or more days per week and the flexibility to work outside the office location for the other days.