Extreme Value Theory (EVT) provides statistical inference tools to model the behavior of random variables under extreme conditions, typically when the norm of a covariate exceeds a high threshold. In many contexts (covariate-shifts, climate change), extrapolation (or out-of-sample) properties of the predictors constructed in a supervised learning framework are crucial, and obtaining good generalization properties on unobserved regions of the covariate space is key. Recent work has explored learning of angular predictors from the most extreme observations under regular variation assumptions, in settings such as binary classification and least squares regression with empirical risk minimization. We extend this framework to penalized learning procedures, focusing on SVM-based quantile regression. We establish finite-sample learning guarantees and illustrate our results on a real life multivariate dataset involving riverflow measurement on the Danube river network.