Return to flip book view

QUSANDBOX THE NEW DECALOGUE ML GOVERNANCE IN THE AGE OF DATA SCIENCE AND AI WWW QUANTUNIVERSITY COM OUR SERVICES Model Governance and Algorithmic Audits Model Life Cycle Management AI ML Onboarding Third party Model Validation Training and Education QuSandbox ABOUT US QuantUniversity is a quantitative analytics and model risk advisory based in Boston MA We provide Quant Finance Data Science and Machine Learning based solutions focused on model risk made accessible through QuSandbox QuantUniversity has worked with analysts and executives from Bloomberg Fidelity Ford Goldman Sachs IBM J P Morgan Chase Nataxis Global Advisors Pan Agora T D Securities and other institutions providing quantitative advisory services analytics training and model risk solutions Contact us at www quantuniversity com INFO QUSANDBOX COM 10 THINGS YOU NEED TO KNOW ABOUT MODEL GOVERNANCE FOR AI ML MODELS 1 DEFINING MODELS Models are not just restricted to code and associated parameters You have to factor data the programming environment and packages parameters and hyperparameters along with the model code 2 GOVERNING MACHINE LEARNING MODELS WORKFLOW You could have hundreds of machine learning models working alongside traditional models A comprehensive framework is needed to factor the nuances of machine learning models workflows in your governance process 3 MODEL VERIFICATION AND VALIDATION OF MACHINE LEARNING MODELS It s not just sufficient to verify if machine learning models work with historical test validation datasets from a technical perspective You have to validate if the models can be used for business decision making 4 PERFORMANCE METRICS AND EVALUATION CRITERIA The choice of performance metrics and evaluation criteria depends on how the models would be used and for what purpose Evaluate the choices carefully 5 MODEL INVENTORY AND TRACKING Avoid model clutter by having a formal model inventory and tracking system You need to track models data snapshots parameters hyperparameters programming environments etc In addition the entire pipeline needs to be tracked Provenance tracking is important for reproducibility 617 283 7904

QU THE NEW DECALOGUE ML GOVERNANCE IN THE AGE OF DATA SCIENCE AND AI WWW QUANTUNIVERSITY COM 10 THINGS YOU NEED TO KNOW ABOUT MODEL GOVERNANCE FOR AI ML MODELS CONT D 6 DATA GOVERNANCE AND MODEL GOVERANANCE Machine Learning models are by design data driven Integrating Data governance and model governance aspects is essential 7 DEVELOPMENT MODELS VS PRODUCTION MODELS As you design models for inference scalability performance considerations need to be factored Models may have to be redesigned compiled to factor production requirements It is important to test models to ensure production models behave as they were designed 8 FAIRNESS REPRODUCIBILITY AUDITABILITY EXPLAINABILITY INTERPRETABILITY BIAS Depending on the application models should be evaluated to ensure Fairness Reproducibility Auditability Explainability Interpretability Bias considerations are met 9 MACHINE LEARNING CHOICES As the field of machine learning matures you have multiple options Automatic Machine Learning ML as a service Pre trained models and models developed from scratch etc bringing different model governance considerations 10 ROLES AND RESPONSIBILITIES With AI and ML making strides you have many new roles in your model building workflow Data engineers scientists model evaluators cloud engineers DevOps MLOps etc Factor the new roles and define clear responsibilities for all the key stakeholders in the model lifecycle INFO QUSANDBOX COM 617 283 7904