Everything But the Kitchen Sink Feature Generation

This blog is a part of a series on the Data Science Pilot Action Set. In my first blog we introduced the action set and the actions for building data understanding. In this blog we dive into feature generation and selection. 

The Data Science Pilot Action Set is included with SAS Visual Data Mining and Machine Learning (VDMML) and consists of actions that implement a policy-based, configurable, and scalable approach to automating data science workflows. The two actions we will examine in this blog are the featureMachine action and the selectFeatures action.

Data Science Pilot Actions

The featureMachine action not only generates new features, but it also explores the data and screens variables. That means that this action can also take on data exploration. Not only is this action taking on double duty, it can save you time by running task in parallel. When it comes to new features, this action creates everything but the kitchen sink. The list of transformations includes missing indicators, several types of imputation, several types of binning, and much more. Unfortunately, the featureMachine action doesn’t include any subject matter expertise. This action won’t replace domain knowledge in feature generation, but it has everything else covered.

The featureMachine action includes the explorationPolicy and screenPolicy, and the transformationPolicy. The explorationPolicy specifies how the data is grouped together, the screenPolicy controls how much data messiness is acceptable, and the transformationPolicy defines which types of features to create. The resulting output of this action set includes information about the generated features, the features generated using the input data, and an analytic store file for generating the features with new data.

/* Create new features using featureMachine Action */
proc cas;
	loadactionset "dataSciencePilot";
	dataSciencePilot.featureMachine
		/	table 			= "hmeq"
			target 			= "BAD"
			copyVars 		= "BAD"
			explorationPolicy      	= {cardinality = {lowMediumCutoff = 40}}
		    	screenPolicy           	= {missingPercentThreshold=35}
            		transformationPolicy   	= {entropy = True, iqv = True,  kurtosis = True, Outlier = True}
            		transformationOut      	= {name = "TRANSFORMATION_OUT", replace = True}
           		featureOut             	= {name = "FEATURE_OUT", replace = True}
            		casOut                 	= {name = "CAS_OUT", replace = True}
            		saveState              	= {name = "ASTORE_OUT", replace = True}
		;
	run;
quit;

The selectFeatures Action will filter features based on a specified measure. Using the selectionPolicy, you can specify the measure you want to filter on and how many features you want. If no measure is specified, the Mutual Information criterion is used as a default.  In continuation of my coding example, I fed the data generated from the featureMachine action into the selectFeatures action to select the best ten features.

/* Select features using selectFeatures Action */ 
proc cas;
	loadactionset "dataSciencePilot";
	dataSciencePilot.selectFeatures
		/ 	table 		= "CAS_OUT"
			casOut 		= {name = "SELECT_FEATURES_OUT", replace = True}
			target 		= "BAD"
			selectionPolicy = {topk=10}
		;
	run;
quit;

In this blog, we took control over more aspects of the data science workflow. Using the featureMachine action, we were able to create many new features and using the selectFeatures action, we narrowed our features down into a usable number. However, if you are looking for one piece of code that will do it all (or almost it all), stayed tuned for my next blog! In the upcoming blog, we will introduce the dsAutoMl action.