Data Science Lifecycle
A data science lifecycle refers to a set of steps that must be followed while working on any data science problem statement. These steps may vary depending on the company. This lifecycle starts from the collection of raw data to deploying the final model. Hence, following a proper sequence will help us get the utmost information from the raw data and also obtain accurate results.
The data science lifecycle is as follows:
Let’s discuss each phase one by one.
Business Understanding
This is the very first phase of any data science project. The phase is research-oriented. Before working on the given problem statement, it is necessary to understand the domain of the problem. Understanding the factors, working, results, need, objectives, etc of the field becomes a must. That will allow the data scientists to use all the required features to get proper and precise information about the problem statement. Hence, it will help them understand the requirements and make the task easy.
Data Collection
Data collection is the second phase that deals with gathering data from diverse sources and is stored in one place. Data scientists aim to get more data so that more amount of information can be deduced from it. The more amount of data is, more the information can be obtained from it. But, the precaution should be taken that the data sources are authenticated as the correctness of data is essential for accurate results. The data collection sources can be:
- Surveys(Online/ Offline)
- Online tracking
- Business reports and many more
Once the data is collected, the people working on it must understand the meaning of the data. As they are aware of the business, it becomes easy to identify the necessary features and their importance.
Exploratory Data Analysis
EDA(Exploratory Data Analysis) is one of the most important phases. In this phase, the data analysis is performed to convert the obtained data into a structured format so that working on the data becomes easy. This is the initial investigation performed to improve the quality of the data by performing the following:
- Handling missing values
- Handling noisy data
- Handling outliers
- Data transformation
- Data visualization and many more.
Feature Selection
After EDA, the dataset is in a structured format. Then comes the feature selection phase. Here, the features from the dataset are classified as independent (y, the output variable) and dependent variables (X, the input variables). There can be n number of input variables in a given dataset. But, not all are necessary to get accurate results. Hence, we apply some feature selection algorithms to identify the necessary features among all the features. Some of the feature selection algorithms are:
- Filter methods
- Wrapper methods
- Embedded methods
- Hybrid method
Data Modeling
Data modeling is the phase where actual logic is applied to the input variables or the features to obtain some output. The logic used is in the form of machine learning models. Depending upon the problem statement and the required output, data scientists choose suitable models. The variables fit into the models, and output is observed. Some of the machine learning models are as follows:
- Supervised
- Unsupervised
- Reinforcement
Model evaluation
In the model evaluation phase, trained models and their outputs are observed using some parameters. The accuracy, error, precision, etc are taken into consideration. There are various ways to evaluate the model. This phase allows us to understand the models picked and find ways to improve the working of those models. Depending on the models used, there are different parameters or ways to evaluate a given algorithm.
Model deployment
The output of the model evaluation phase will be a final model with utmost accuracy. This model needs to be deployed before presenting it in front of the company. In the real world, the company representatives are not interested in the model used and the coding part instead, they need outputs satisfying their requirements. The finalized model is represented using some front end. The user can give the values to the input parameters and get the output. Hence, the complete model can be deployed using frameworks such as:
- Streamlit(https://streamlit.io/)
- Flask(https://flask.palletsprojects.com/en/2.2.x/)
With the deployment process, the complete data science project is complete and ready to use. There are many hidden steps between the above mention phases. Those steps will come into picture once we start working on some project.
Hope this help!
Cheers,
Sumedha Zaware
