Collecting, storing and processing data

Data collection means identifying data that is already available or gathering it from the right sources. For a grocery shop's sales data:

• If it is in a diary or register, enter it in digital form, for example in a spreadsheet.
• If it is already digital, say in a CSV (comma separated values) file, use that.
• If nothing is recorded, have software made (for example in Python) to store and retrieve sales data in a CSV file or a database such as MySQL.

• Data is constantly generated: hospitals collect patient data to improve services; malls collect purchase data; the World Bank and IMF collect economic data for forecasts.
• Analysis can reveal patterns: if bedsheets and groceries are often bought together, a mall may display bedsheets near the grocery section.

Data storage keeps data on storage devices so it can be retrieved later: HDD, SSD, CD/DVD, tape drive, pen drive, memory card. Volumes are huge, but cheaper devices help. Images, documents, audio and video are kept as files; school and hospital records as data files. File processing has limitations, which a Database Management System (DBMS) overcomes.

Data processing turns raw data (numbers, text, images) into information (tables, charts, text). The data process cycle:

• Input: data collection, data preparation and data entry.
• Processing: store, retrieve, classify, update.
• Output: reports and results.
• Automated processing is all around us: online bill payment, registering complaints, booking tickets.

The book's three worked tasks (Figure 5.2): an exam website checks applicants' details and fee payment and outputs an admit card with roll number and centre; a bank's ATM checks the PIN and balance, deducts the amount and outputs cash and a slip; a railway system checks login and berth availability, takes payment and outputs a ticket with PNR and berth (or a waiting number).

Example (new): for a school library: input is the student's card number and the book's barcode; processing checks that the student has no overdue book and records the issue date; output is a due-date slip and an updated list of books on loan.