A comprehensive guide for data scientists to master effective data cleaning tools and techniques

Key Features:

• Think about your data intelligently and ask the right questions
• Master data cleaning techniques using hands-on examples belonging to diverse domains
• Work with detailed, commented, well-tested code samples in Python and R

Book Description:

In data science, data analysis, or machine learning, most of the effort needed to achieve your actual purpose lies in cleaning your data. Using Python, R, and command-line tools, you will learn the essential cleaning steps performed in every production data science or data analysis pipeline. This book not only teaches you data preparation but also what questions you should ask of your data.

The book dives into the practical application of tools and techniques needed for data ingestion, anomaly detection, value imputation, and feature engineering. It also offers long-form exercises at the end of each chapter to practice the skills acquired.

You will begin by looking at data ingestion of a range of data formats. Moving on, you will impute missing values, detect unreliable data and statistical anomalies, and generate synthetic features that are necessary for successful data analysis and visualization goals.

By the end of this book, you will have acquired a firm understanding of the data cleaning process necessary to perform real-world data science and machine learning tasks.

What You Will Learn:

• Ingest and work with common tabular, hierarchical, and other data formats
• Apply useful rules and heuristics for assessing data quality and detecting bias
• Identify and handle unreliable data and outliers in their many forms
• Impute sensible values into missing data and use sampling to fix imbalances
• Generate synthetic features that help to draw out patterns in your data
• Prepare data competently and correctly for analytic and machine learning tasks

Who this book is for:

This book is designed to benefit software developers, data scientists, aspiring data scientists, and students who are interested in data analysis or scientific computing. Basic familiarity with statistics, general concepts in machine learning, knowledge of a programming language (Python or R), and some exposure to data science are helpful. The text will also be helpful to intermediate and advanced data scientists who want to improve their rigor in data hygiene and wish for a refresher on data preparation issues.

Move Beyond Python Code That Mostly Works to Code That Is Expressive, Robust, and Efficient

Python is arguably the most-used programming language in the world, with applications from primary school education to workaday web development, to the most advanced scientific research institutes. While there are many ways to perform a task in Python, some are wrong, inelegant, or inefficient. Better Python Code is a guide to Pythonic programming, a collection of best practices, ways of working, and nuances that are easy to miss, especially when ingrained habits are borrowed from other programming languages.

Author David Mertz presents concrete and concise examples of various misunderstandings, pitfalls, and bad habits in action. He explains why some practices are better than others, based on his 25+ years of experience as an acclaimed contributor to the Python community. Each chapter thoroughly covers related clusters of concepts, with chapters sequenced in ascending order of sophistication.

Whether you are starting out with Python or are an experienced developer pushing through the limitations of your Python code, this book is for all who aspire to be more Pythonic when writing better Python code.

• Use the right kind of loops in Python
• Learn the ins and outs of mutable and immutable objects
• Get expert advice to avoid Python gotchas
• Examine advanced Python topics
• Navigate the attractive nuisances that exist in Python
• Learn the most useful data structures in Python and how to avoid misusing them
• Avoid security mistakes
• Understand the basics of numeric computation, including floating point numbers and numeric datatypes

My high expectations for this engaging Python book have been exceeded: it offers a great deal of insight for intermediate or advanced programmers to improve their Python skills, includes copious sharing of precious experience practicing and teaching the language, yet remains concise, easy to read, and conversational.
--From the Foreword by Alex Martelli

Register your book for convenient access to downloads, updates, and/or corrections as they become available. See inside book for details.

Learn how AI-assisted coding using ChatGPT and GitHub Copilot can dramatically increase your productivity (and fun) writing regular expressions and other programs.

Regular Expression Puzzles and AI Coding Assistants is the story of two competitors. On one side is David Mertz, an expert programmer and the author of the Web's most popular Regex tutorial. On the other are the AI powerhouse coding assistants, GitHub Copilot and OpenAI ChatGPT.

Here's how the contest works: David invents 24 Regex problems he calls puzzles and shows you how to tackle each one. When he's done, he has Copilot and ChatGPT work the same puzzles. What they produce intrigues him. Which side is likelier to get it right? Which will write simple and elegant code? Which makes smarter use of lesser known Regex library features? Read the book to find out.

David also offers AI best practices, showing how smart prompts return better results. By the end, you'll be a master at solving your own Regex puzzles, whether you use AI or not.

Purchase of the print book includes a free eBook in PDF, Kindle, and ePub formats from Manning Publications.

About the technology
Groundbreaking large language model research from OpenAI, Google, Amazon, and others have transformed expectations of machine-generated software. But how do these AI assistants, like ChatGPT and GitHub Copilot, measure up against regular expressions--a workhorse technology for developers used to describe, find, and manipulate patterns in text. Regular expressions are compact, complex, and subtle. Will AI assistants handle the challenge?

About the book
Regular Expression Puzzles and AI Coding Assistants is the perfect starting point for programmers of any experience level who want to understand the capabilities--and the limitations--of these exciting new tools. Author David Mertz presents 24 challenging regex puzzles, their traditional human-made solutions, and the fascinating answers given by popular AI assistants. Alongside these eye-opening puzzles you'll learn how to write prompts, integrate AI-generated coding suggestions, and interact with the assistant to get the results you want. By the end of the book, you'll have a clear understanding of where AI assistants can reliably write code for you and where you'll still need a human touch. Plus, you'll learn a lot about regular expressions!

About the reader
Code examples use simple Python and Regular Expressions. No experience with AI coding tools required.

About the author
David Mertz is the founder of KDM Training and an acclaimed contributor to the Python community. He is also the author of The Puzzling Quirks of Regular Expressions, Cleaning Data for Effective Data Science: Doing the Other 80% of the Work, and other books.

Table of Contents
1 The map and the territory
2 Quantifiers and special sub-patterns
3 Pitfalls and sand in the gears
4 Creating functions using regex
5 Easy, difficult, and impossible tasks
6 Conclusions
Appendix A: Learning to use regular expressions

This entertaining puzzle book, for software developers and programming hobbyists, enlightens readers about many of the amazing and surprising behaviors of regular expressions. The author presents a series of questions, each inviting readers to think at length--and indeed to try out code on their own--before turning the page for the author's discussion and solution. The code shown in this book gives attention to regular expressions themselves, but is run using the Python programming language in particular. Everything discussed can easily be adapted to other programming languages. The short book is accompanied by quirky and obliquely relevant illustrations that add character and whimsy to the text.