I am talking about Python Pandas .agg() function, this one:
meanData = all_data.groupby(['Id'])[features].agg('mean') So, it can do things like:
- Mean
- Median
- Sum
- Max
- Min
- Std
What else can it do? I found nothing on the official documentation page:
5 Answers
You can find the full list in documentation under pandas.core.groupby.GroupBy.some-function-name in the left menu.
List currently includes many aggregation functions:
pipe, all, any, bfill, backfill, count, cumcount, cummax, cummin, cumprod, cumsum, ffill, first, head, last, max, mean, median, min, ngroup, nth, ohlc, pad, prod, rank, pct_change, size, sem, std, sum, var, tail
The aggregations that work with .agg() include:
Mean - df.agg('mean')
Median - df.agg('median')
Mode - df.agg('mode')
Sum - df.agg('sum')
Count - df.agg('count')
Max - df.agg('max')
Min - df.agg('min')
Standard Deviation - df.agg('std')
Variance - df.agg('var')
Skewness - df.agg('skew')
Kurtosis - df.agg('kurt')
The relevant parts of pandas/core/base.py (here, line 298):
def _try_aggregate_string_function(self, arg, *args, **kwargs): """ if arg is a string, then try to operate on it: - try to find a function (or attribute) on ourselves - try to find a numpy function - raise """ f = getattr(self, arg, None) if f is not None: if callable(f): return f(*args, **kwargs) f = getattr(np, arg, None) if f is not None: return f(self, *args, **kwargs) raise ValueError("{arg} is an unknown string function".format(arg=arg)) Essentially it tries to introspect using the string as a function, and then tries the same with numpy, in case it's a builtin. If not it returns a ValueError.
I'd be happy if someone who knows more than me could clarify more, but if not, hope this helps.
It can be just about any function that can be applied on a DataFrame object.
print(dir(DataFrame)) When func is a string type, the func name is looked up in the available attributes of the DataFrame object that the .agg method is invoked on.
While it gives similar result when doing a division of elements in a DataFrame to write,
df = DataFrame([1,2,3,4]) df.agg('true_div', 0, 2) In the real world, you find that the method doing the operation is invoked in a direct manner on the DataFrame
df = DataFrame([1,2,3,4]) df.true_div(2) The answers here and here give what you are looking but I want to add one of agg usage from one of my experience. I created a manual class to call as a function and use it in agg to show flexibility of agg -->
The class:
class Quantile: def __init__(self, the_quantile): self.the_quantile = the_quantile def __call__(self, the_array): return np.quantile(the_array.dropna(), self.the_quantile) The agg code block I used it in:
df.groupby('SomeGroups')['SomeValues'] \ .agg(count = 'count', nunique = 'nunique', size = 'size', minimum = 'min', idx_min = 'idxmin', mean = 'mean', median = 'median', maximum = 'max', idx_max = 'idxmax', std = 'std', mad = 'mad', quantile_0 = Quantile(0), quantile_25 = Quantile(0.25), quantile_50 = Quantile(0.5), quantile_57 = Quantile(0.57), #This was to show I can call any quantile quantile_90 = Quantile(0.9), quantile_95 = Quantile(0.95), quantile_98 = Quantile(0.98), quantile_100 = Quantile(1)) 
