PDF/EPUB Weapons of Math Destruction : How Big Data Increases Inequality and Threatens Democracy

Weapons of Math Destruction : How Big Data Increases Inequality and Threatens Democracy

download ï E-book, or Kindle E-pub ✓ Cathy O Neil

BOMB PARTS What Is a Model It was a hot August afternoon in Lou Boudreau the player manager of the Cleveland Indians was having a miserable day In the first game of a doubleheader Ted Williams had almost single handedly annihilated his team Williams perhaps the games greatest hitter at the time had smashed three home uns and driven home eight The Indians ended up losing to Boudreau had to take action So when Williams came up for the first time in the second game players on the Indians side started moving around Boudreau the shortstop jogged over to where the second baseman would usually stand and the second baseman backed into short How Florida Happened: The Political Education of Buddy MacKay right field The third baseman movedto his left into the shortstops hole It was clear that Boudreau perhaps out of desperation was shifting the entire orientation of his defense in an attempt to turn Ted Williamss hits into outs In other words he was thinking like a data scientist He had analyzed crude data most of it observational Ted Williams usually hit the ball toight field Then he adjusted And it worked Fielders caughtof Williamss blistering line drives than before though they could do nothing about the home Narratives of Human Evolution runs sailing over their heads If you go to a major league baseball game today youll see that defenses now treat nearly every player like Ted Williams While Boudreau merely observed where Williams usually hit the ball managers now know precisely where every player has hit every ball over the last week over the last month throughout his career against left handers when he has two strikes and so on Using this historical data they analyze their current situation and calculate the positioning that is associated with the highest probability of success And that sometimes involves moving players far across the field Shifting defenses is only one piece of a much larger uestion What steps can baseball teams take to maximize the probability that theyll win In their hunt for answers baseball statisticians have scrutinized every variable they can uantify and attached it to a value How muchis a double worth than a single When if ever is it worth it to bunt aunner from first to second base The answers to all of these uestions are blended and combined into mathematical models of their sport These are parallel universes of the baseball world each a complex tapestry of probabilities They include every measurable Understanding Autism in Adults and Aging Adults: Improving Diagnosis and Quality of Life relationship among every one of the sports components from walks to homeuns to the players themselves The purpose of the model is to Report on Radiation and the Quantum-Theory, 1914 run differentscenarios at every juncture looking for the optimal combinations If the Yankees bring in aight handed pitcher to face Angels slugger Mike Trout as compared to leaving in the current pitcher how muchlikely are they to get him out And how will that affect their overall odds of winning Baseball is an ideal home for predictive mathematical modeling As Michael Lewis wrote in his bestseller Moneyball the sport has attracted data nerds throughout its history In decades past fans would pore over the stats on the back of baseball cards analyzing Carl Yastrzemskis home Lioness run patterns or comparing Roger Clemenss and Dwight Goodens strikeout totals But starting in the s serious statisticians started to investigate what these figures along with an avalanche of new oneseally meant how they translated into wins and how executives could maximize success with a minimum of dollars Moneyball is now shorthand for any statistical approach in domains long What Do Grown-ups Do All Day? ruled by the gut But baseballepresents a healthy case studyand it serves as a useful contrast to the toxic models or WMDs that are popping up in so many areas of our lives Baseball models are fair in part because theyre transparent Everyone has access to the stats and can understandor less how theyre interpreted Yes one teams model might givevalue to home Non Censure run hitters while another might discount them a bit because sluggers tend to strike out a lot But in either case the numbers of homeuns and strikeouts are there for everyone to see Baseball also has statistical Herrin der Schmuggler rigor Its gurus have an immense data set at hand almost all of it directlyelated to the performance of players in the game Moreover their data is highly Hope and Glory relevant to the outcomes they are trying to predict This may sound obvious but as well see throughout this book the folks building WMDsoutinely lack data for the behaviors theyre most interested in So they substitute stand in data or proxies They draw statisticalcorrelations between a persons zip code or language patterns and her potential to pay back a loan or handle a job These correlations are discriminatory and some of them are illegal Baseball models for the most part dont use proxies because they use pertinent inputs like balls strikes and hits Most crucially that data is constantly pouring in with new statistics from an average of twelve or thirteen games arriving daily from April to October "Statisticians can compare the esults of these games to the predictions of their "can compare the esults of these games to the predictions of their and they can see where they were wrong Maybe they predicted that a left handed Hysterie reliever would give up lots of hits toight handed battersand yet he mowed them down If so the stats team has to tweak their model and also carry out Der Flügel des Engels : Autobiogr., Selbstzeugnisse, Briefe an Jack Kerouac research on why they got it wrong Did the pitchers new screwball affect his statistics Does he pitch better at night Whatever they learn they can feed back into the modelefining it Thats how trustworthy models operate They maintain a constant back and forth with whatever in the world theyre trying to understand or predict Conditions change and so must the model Now you may look at the baseball model with its thousands of changing variables and wonder how we could even be comparing it to the model used to evaluate teachers in Washington DC schools In one of them an entire sport is modeled in fastidious detail and updated continuously The other while cloaked in mystery appears to lean heavily on a handful of test داستان‌های کوتاه امریکای لاتین results from one year to the next Is thateally a model The answer is yes A model after all is nothingthan an abstract epresentation of some process be it a baseball game all is nothingthan an abstract epresentation of some process be it a baseball game oil companys supply chain a foreign governments actions or a movie theaters attendance Whether its Love and Its Place in Nature: A Philosophical Interpretation of Freudian Psychoanalysis running in a computer program or in our head the model takes what we know and uses it to predictesponses in various situations All of us carry thousandsof models in our heads They tell us what to expect and they guide our decisions Heres an informal model I use every day As a mother of three I cook the meals at homemy husband bless his cannot emember to put salt in pasta water Each night when I begin to cook a family meal I internally intuitively model everyones appetite I know that one of my sons loves chicken but hates hamburgers while another will eat only the pasta with extra grated parmesan cheese But I also have to take into account that peoples appetites vary from day to day so a change can catch my model by surprise Theres some unavoidable uncertainty involved The input to my internal cooking model is the information I have about my family the ingredients I have on hand or I know are available and my own energy time and ambition The output is how and what I decide to cook I evaluate the success of a meal by how satisfied my family seems at the end of it how much theyve eaten and how healthy the food was Seeing how well it is eceived and how much of it is enjoyed allows me to update my model for the next time I cook The updates and adjustments make it what statisticians call a dynamic model Over the years Ive gotten pretty good at making meals for my family Im proud to say But what if my husband and I go away for a week and I want to explain my system to my mom so she can fill in for me Or what if my friend who has kids wants to know my methods Thats when Id start to formalize my model making it muchsystematic and in some sense mathematical And if I were feeling ambitious I might put it into a computer program Ideally the program would include all of the available food options their nutritional value and cost and a complete database of my familys tastes each individuals preferences and aversions It would be hard though to sit down and summon all thatinformationoff the top of my head Ive got loads of memories of people grabbing seconds of asparagus or avoiding the string beans But theyre. ,
All mixed up and hard to formalize in a comprehensive list The better solution would be to train the model over time entering data every day on what Id bought and cooked and noting the Triskell Tales: Twenty-Two Years of Chapbooks responses of each family member I would also include parameters or constraints I might limit the fruits and vegetables to whats in season and dole out a certain amount of Pop Tarts but only enough to forestall an openebellion I also would add a number of Across the Wide and Lonesome Prairie rules This one likes meat this one likes bread and pasta this one drinks lots of milk and insists on spreading Nutella on everything in sight If I made this work a major priority over many months I might come up with a very good model I would have turned the food management I keep in my head my informal internal model into a formal external one In creating my model Id be extending my power and influence in the world Id be building an automated me that others can implement even when Im not around There would always be mistakes however because models are by their very nature simplifications No model can include all of theeal worlds complexity or the nuance of human communication Inevitably some important information gets left out I might have neglected to inform my model that junk food Mein wildes Geheimnis 03 rules areelaxed on birthdays or that Ein Goldfisch räumt auf raw carrots arepopular than the cooked variety To create a model then we make choices about whats important enough to include simplifying the world into a toy version that can be easily understood and from which we can infer important facts and actions We expect it to handle only one job and accept that it will occasionally act like a clueless machine one with enormous blind spots Sometimes these blind spots dont matter When we ask Google Maps for directions it models the world as a series ofoads tunnels and bridges It ignores the buildings because they arent elevant to the task When avionics software guides an airplane it models the wind the speed of the plane and the landing strip below but not the streets tunnels buildings and people A models blind spots eflect the judgments and priorities of its creators While the choices in Google Maps and avionics software appear cut and dried others are farproblematic The value added model in Washington DC schools to eturn to that example evaluates teachers largely on the basis of students test scores while ignoring how much the teachers engage the students work on specific skills deal with classroom management or help students with personal and family problems Its overly simple sacrificing accuracy and insight for efficiency Yet from the administrators perspective it provides an effective tool to ferret out hundreds of apparently underperforming teachers even at the isk of misreading some of them Here we see that models despite their A London Home in the 1890s reputation for impartialityeflect goals and ideology When I The Geology And Landscape Of Santa Barbara County, California, And Its Offshore Islands removed the possibility of eating Pop Tarts at every meal I was imposing my ideology on the meals model Its something we do without a second thought Our own values and desires influence our choices from the data we choose to collect to the uestions we ask Models are opinions embedded in mathematics Whether or not a model works is also a matter of opinion After all a key component of every model whether formal or informal is its definition of success This is an important point that welleturn to as we explore the dark world of WMDs In each case we must ask not only who designed the model but also what that person or company is trying to accomplish If the North Korean government built a model for my familys meals for example itmight be optimized to keep us above the threshold of starvation at the lowest cost based on the food stock available Preferences would count for little or nothing By contrast if my kids were creating the model success might feature ice cream at every meal My own model attempts to blend a bit of the North Koreans esource management with the happiness of my kids along with my own priorities of health convenience diversity of experience and sustainability As a esult its muchcomplex But it still Jane Doe and the Cradle of All Worlds reflects my own personaleality And a model built for today will work a bit worse tomorrow It will grow stale if its not constantly updated Prices change as do peoples preferences A model built for a six year old wont work for a teenager This is true of internal models as well You can often see troubles when grandparents visit a grandchild they havent seen for a while On their previous visit they gathered data on what the child knows what makes her laugh and what TV show she likes and unconsciously created a model for Is That Even a Country, Sir! relating to this particular four year old Upon meeting her a year later they can suffer a few awkward hours because their models are out of date Thomas the Tank Engine it turns out is no longer cool It takes some time to gather new data about the child and adjust their models This is not to say that good models cannot be primitive Some very effective ones hinge on a single variable The most common model for detecting fires in a home or office weighs only one strongly correlated variable the presence of smoke Thats usually enough But modelersun into problemsor subject us to problemswhen they focus models as simple as a smoke alarm on their fellow humans Racism at the individual level can be seen as a predictive model whirring away in billions of human minds the individual level can be seen as a predictive model whirring away in billions of human minds the world It is built from faulty incomplete or generalized data Whether it comes from experience or hearsay the data indicatesthat certain types of people have behaved badly That generates a binary prediction that all people of that うそつきリリィ 3 [Usotsuki Lily 3] race will behave that same way Needless to sayacists dont spend a lot of time hunting down The Prince reliable data to train their twisted models And once their model morphs into a belief it becomes hardwired It generates poisonous assumptions yetarely tests them settling instead for data that seems to yet A Bold Carnivore: An Alphabet of Predators rarely tests them settling instead for data that seems to and fortify them Conseuentlyacism is the most slovenly of predictive models It is powered by haphazard data gathering and spurious correlations Goodbye Sarajevo: A True Story of Courage, Love and Survival reinforced by institutional ineuities and polluted by confirmation bias In this way oddly enoughacism operates like many of the WMDs Ill be describing in this book Ce texte fait Lifting r fence l dition Brochew York Times Book Review Notable Book of A Boston Globe Best Book of One of Wired s Reuired Reading Picks of One of Fortune s Favorite Books of A Kirkus ReviewsBest Book of A Chicago Public Library Best Book of A Nature Best Book of An On PointBest Book of New York Times Editor s ChoiceA Maclean s BestsellerWinner of the SLA NY PrivCo Spotlight AwardONeils book offers a frightening look at how algorithms are increasingly Mathruhridayam regulating people Her knowledge of the power andisks of mathematical models coupled with a gift for analogy makes her one of the most valuable observers of the continuing weaponization of big data She does a masterly job explaining the pervasiveness and isks of the algorithms that egulate our lives New York Times Book Review Weapons of Math Destruction is the Big Data story Silicon Valley proponents won t tell It pithily exposes flaws
in how information 
how information used to assess everything from creditworthiness to policing tactics a thought provoking A Life In School: What The Teacher Learned read for anyone inclined to believe that data doesn t lie ReutersThis is a manual for the st century citizen and it succeeds where other big data accounts have failed it is accessibleefreshingly critical and feels elevant and urgent Financial Times Insightful and disturbing New York Review of Books Weapons of Math Destruction is an urgent critiue of the ampant misuse of math in nearly every aspect of our livesBoston GlobeA fascinating and deeply disturbing book Yuval Noah Harari author of Sapiens The Guardians Best Books of Illuminating ONeil makes a convincing case that this The House That Had Enough reliance on algorithms has gone too farThe AtlanticA nuancedeminder that big data is only as good as the people wielding itWiredIf youve ever suspected there was something baleful about our deep trust in data but lacked the mathematical skills to figure out exactly what it was this is the book for you SalonONeil is an ideal person to write this book She is an academic mathematician turned Wall Street uant turned data scientist who has been involved in Occupy Wall Street and The Devil's Snake Curve: A Fan's Notes From Left Field recentlystarted an algorithmic auditing company She is on. E of the strongest voices speaking out for limiting the ways we allow algorithms to influence our livesWhile Weapons of Math Destructionis full of hard truths and grim statistics it is also accessible and even entertaining ONeils writing is direct and easy toeadI devoured it in an afternoon Scientific AmericanReadable and engaging succinct and cogent Weapons of Math Destruction is The Jungle of our age It should be The March Up: Taking Baghdad with the 1st Marine Division reuiredeading for all data scientists and for any organizational decision maker convinced that a mathematical model can Pimpinella Meerprinzessin 1: Ankunft im Muschelschloss (German Edition) replace human judgment Mark Van Hollebeke Data and Society PointsIndispensable Despite the technical complexity of its subject Weapons of Math Destruction lucidly guideseaders through these complex modeling systems ONeils book is an excellent primer on the ethical and moral Das kurze Leben der Sophie Scholl risks of Big Data and an algorithmically dependent world For those curious about how Big Data can help them and their businesses or how it has beeneshaping the world around them Weapons of Math Destruction is an essential starting placeNational PostCathy ONeil has seen Big Data from the inside and the picture isnt pretty Weapons of Math Destructionopens the curtain on algorithms that exploit people and distort the truth while posing as neutral mathematical tools This book is wise fierce and desperately necessaryJordan Ellenberg University of Wisconsin Madison author of How Not To Be WrongONeil has become a whistle blower for the world of Big Data in her important new book Her work makes particularly disturbing points about how being on the wrong side of an algorithmic decision can snowball in incredibly destructive ways TIMEONeils work is so important her book is a vital crash course in the specialized kind of statistical knowledge we all need to interrogate the systems around us and demand betterBoing BoingCathy ONeil a number theorist turned data scientist delivers a simple but important message Statistical models are everywhere and they exert increasing power over many aspects of our daily lives Weapons of Math Destruction provides a handy map to a few of the many areas of our lives over which invisible algorithms have gained some control As the empire of big data continues to expand Cathy ONeils Défendre Jacob reminder of the need for vigilance is welcome and necessary American ProspectAn avowed math nerd ONeil has written an engaging description of the effect of crunched data on our lives Hicklebees San Francisco ChronicleBy tracking how algorithms shape people s lives at every stage O Neil makes a compelling case that our bot overlords are using data to discriminate unfairly and foreclose democratic choices If you work with data or just produceeams of it online this is a must Darfur's Sorrow: A History of Destruction and Genocide read ArsTechnica Lucid alarming and valuable ONeils writing is crisp and precise as she aims her arguments to a lay audience This makes for aemarkably page turning Black Gold in North Dakota read for a book about algorithms Weapons of Math Destruction should beeuired eading for anybody whose life will be affected by Big Data which is to say euired Nestle Toll House Recipe Collection reading for everyone Its a wake up call a journalistic heir to The Jungle and Silent Spring Like those books it should change the course of American society Aspen Times O Neil s propulsive studyeveals many models that are currently micromanaging the US economy as opaue and iddled with bias NatureYou dont need to be a nerd to appreciate the significance of ONeils message Weapons is a must ead for anyone who is working to combat economic and Wicked City: The Other Side racial discriminationGoop Cathy ONeils book is important and covers issues everyone should care about Bonus points its accessible compelling and something I wasnt expectingeally fun to Officer, Surgeon…Gentleman! read Inside Higher EdOften we dont even know where to look for those important algorithms because by definition the most dangerous ones are also the most secretive Thats why the catalogue of case studies in ONeils book are so important shes telling us where to look The GuardianONeil is passionate about exposing the harmful effects of Big Datadriven mathematical models what she calls WMDs and shes uniuely ualified for the task She makes a convincing case that many mathematical models today are engineered to benefit the powerful at the expense of the powerless and has written an entertaining and timely book that giveseaders the tools to cut through the ideological fog obscuring the dangers of the Big Data Barefoot through Mauretania revolution In These TimesIn this simultaneously illuminating and disturbing account ONeil describes the many ways in which widely used mathematic modelsbased on prejudice misunderstanding and biastend to punish the poor andeward the Dude, You're a Fag: Masculinity and Sexuality in High School rich She convincingly argues for bothresponsible modeling and federalegulation An unusually lucid and eadable look at the daunting algorithms that govern so unusually lucid and eadable look at the daunting algorithms that govern so aspects of our livesKirkus Reviewsstarred Even As A Professional Mathematician as a professional mathematician had no idea how insidious Big Data could be until I ead Weapons of Math Destruction Though terrifying its a surprisingly fun ead ONeils vision of a world Colorblind run by algorithms is laced with dark humor and exasperationlike a modern day Dr Strangelove or Catch It is eye opening disturbing and deeply importantSteven Strogatz Cornell University author of The Joy of xThis taut and accessible volume the stuff of technophobes nightmares explores the myriad ways in which largescale data modeling has made the world a less just and eual place ONeil speaks from a place of authority on the subject Unlike some otherecent books on data collection hers is not hysterical she offersof a chilly wake up call as she walks eaders through the ways the big data industry has facilitated social ills such as skyrocketing college tuitions policing based on acial profiling and high unemployment ates in vulnerable communities eerily prescient Publishers Weekly Well written entertaining and very valuable Times Higher Education Not math heavy but written in an exceedingly accessible almost literary style O Neil s fascinating case studies of WMDs fit neatly into the genre of dystopian literature There s a little Philip K Dick a little Orwell a little Kafka in her portrait of powerful bureaucracies ceding control of the most intimate decisions of our lives to hyper empowered computer models iddled with all of our unresolved atavistic human biases Paris ReviewThrough harrowing eal world examples and lively story telling Weapons of Math Destruction shines invaluable light on the invisible algorithms harrowing eal world examples and lively story telling Weapons of Math Destruction shines invaluable light on the invisible algorithms complex mathematical models used by government and big business to undermine euality and increase private power Combating secrecy with clarity and confusion with understanding this book can help us change course before its too lateAstra Taylor author of The Peoples Platform Taking Back Power and Culture in the Digital AgeWeapons of Math Destructionis a fantastic plainspoken call to arms It acknowledges that models aren t going away As a tool for identifying people in difficulty they are amazing But as a tool for punishing and disenfranchising they Classic Sunil Gangopadhyay re a nightmareCory Doctorow author of Little Brother and co editor of Boing BoingMany algorithms are slaves to the ineualities of power and prejudice If you dont want these algorithms to become your mastersead Weapons of Math Destruction by Cathy ONeil to deconstruct the latest growing tyranny of an arrogant establishmentRalph Nader author of Unsafe at Any SpeedIn this fascinating account Cathy O Neil leverages her expertise in mathematics and her passion for social justice to poke holes in the triumphant narrative of Big Data She makes a compelling case that math is being used to sueeze marginalized segments of society and magnify ineuities Her analysis is superb her writing is enticing and her findings are unsettlingdanah boyd founder of Data Society and author of Its ComplicatedFrom getting a job to finding a spouse predictive algorithms are silently shaping and controlling our destinies Cathy O Neil takes us on a journey of outrage and wonder with prose that makes you feel like it s just a conversation But its an important one We need to The Unemployed Guy's Guide to Unemployment reckon with technology Linda Tirado author ofHand to Mouth Living in Bootstrap AmericaNext time you hear someone gushing uncritically about the wonders of Big Data show them Weapons of Math Destruction Itll be salutaryFelix Salmon Fusion Ce texte fait f The Patriots Club rence l dition Broch.