TIL of #conformalprediction, a way to assess the uncertainty of a prediction (from any algorithm, including from #machineleaning). It is used in research to make #autonomousdriving safer by predicting other agent's movements: https://www.youtube.com/watch?v=QvIJH4cZy3E

It does not require an expert model, but in turn it needs a statistically representative dataset.

RT @predict_addict@twitter.com

The Key Invited talk by Prof Emmanuel Candes (Stanford) on “Conformal Prediction in 2022” it is now available online to watch.

https://slideslive.com/38996063/conformal-prediction-in-2022?ref=speaker-43789

#conformalprediction #machinelearning

🐦🔗: https://twitter.com/predict_addict/status/1613862324653395973

Why perform cross validation (CV) in #MachineLearning? To estimate the generalization error of a trained predictor. This paper uses the idea of a #ProperLoss (called Q-class). Then it covers CV, bootstrap, and Mallow's covariance penalties. It also covers #ConformalPrediction, which is newly popular because of Emanuel Candes' keynote at #NeurIPS 2022
https://doi.org/10.3390/stats4040063
The paper is also a good advertisement for Efron and Hastie's recent book.

🚀 #AWS Fortuna is skyrocketing! 🚀 Just a few days, and so many GitHub stars and forks! ⭐️

Fortuna supports #ConformalPrediction, #BayesianInference and other methods for #UncertaintyQuantification in #DeepLearning.

Try it out and let us know!
https://github.com/awslabs/fortuna

In collaboration with @cedapprox @andrewgwils and team.

#uncertainty #neuralnetworks #bayesian #conformal #calibration #jax #flax #python #opensource #library #machinelearning #ai

🚀 #AWS Fortuna is skyrocketing! 🚀 Just a few days, and so many GitHub stars and forks! ⭐️

Fortuna supports #ConformalPrediction, #BayesianInference and other methods for #UncertaintyQuantification in #DeepLearning.

Try it out and let us know!
https://github.com/awslabs/fortuna

In collaboration with @cedapprox, @andrewgwils and team.

#uncertainty #neuralnetworks #bayesian #conformal #calibration #jax #flax #python #opensource #library #machinelearning #ai

🚀 #AWS Fortuna is skyrocketing! 🚀 Just a few days, and so many GitHub stars and forks! ⭐️

Fortuna supports #ConformalPrediction, #BayesianInference and other methods for #UncertaintyQuantification in #DeepLearning.

Try it out and let us know!
https://github.com/awslabs/fortuna

In collaboration with @cedapprox, @andrewgwils and team.

#uncertainty #neuralnetworks #bayesian #conformal #calibration #jax #flax #python #opensource #library #machinelearning #ai

Today, we open sourced Fortuna (https://github.com/awslabs/fortuna) a library for uncertainty quantification.
Deep neural networks are often overconfident and do not know what they don’t know. Quantifying the uncertainty in the predictions they make will help deploy deep learning more responsibly and more safely.
#responsibleAI #ConformalPrediction #BayesianInference #UncertaintyQuantification #deeplearning #opensource

Just shared a more hands-on guide for using the new package for #conformalprediction in #julia that I’ve been working on: https://github.com/pat-alt/ConformalPrediction.jl
“How to Coformalize a Deep Image Classifier” on TDS (https://towardsdatascience.com/how-to-conformalize-a-deep-image-classifier-14ead4e1a5a0) or my blog (https://www.paltmeyer.com/blog/posts/conformal-image-classifier/)

Thoughts and contributions welcome 🤗

One approach to do conformal prediction in regression is to use quantile regression (pinball loss). One annoying thing about quantile regression is that if you estimate multiple quantiles, they could cross (and they really shoudn't). This paper proposes a method that prevents crossing (there are other papers that do so too), in particular for conformal prediction.

https://arxiv.org/abs/2210.10161v1

#MachineLearning #ConformalPrediction #QuantileRegression