How to find the line that is generating a Pandas SettingWithCopyWarning?

Set pd.options.mode.chained_assignment=”raise” This will throw an exception pointing to the line which triggers SettingWithCopyError. UPDATE: how to catch the error, and interrogate the stacktrace to get the actual offending lineno: import pandas as pd from inspect import currentframe, getframeinfo from pandas.core.common import SettingWithCopyError pd.options.mode.chained_assignment=”raise” df = pd.DataFrame({‘a’: [1, 2, 3], ‘b’: [4, 5, 6]}) df2 … Read more

Correct way to set value on a slice in pandas [duplicate]

This is a very common warning from pandas. It means you are writing in a copy slice, not the original data so it might not apply to the original columns due to confusing chained assignment. Please read this post. It has detailed discussion on this SettingWithCopyWarning. In your case I think you can try data.loc[data[‘name’] … Read more

What rules does Pandas use to generate a view vs a copy?

Here’s the rules, subsequent override: All operations generate a copy If inplace=True is provided, it will modify in-place; only some operations support this An indexer that sets, e.g. .loc/.iloc/.iat/.at will set inplace. An indexer that gets on a single-dtyped object is almost always a view (depending on the memory layout it may not be that’s … Read more

why should I make a copy of a data frame in pandas

This expands on Paul’s answer. In Pandas, indexing a DataFrame returns a reference to the initial DataFrame. Thus, changing the subset will change the initial DataFrame. Thus, you’d want to use the copy if you want to make sure the initial DataFrame shouldn’t change. Consider the following code: df = DataFrame({‘x’: [1,2]}) df_sub = df[0:1] … Read more

How to deal with SettingWithCopyWarning in Pandas

The SettingWithCopyWarning was created to flag potentially confusing “chained” assignments, such as the following, which does not always work as expected, particularly when the first selection returns a copy. [see GH5390 and GH5597 for background discussion.] df[df[‘A’] > 2][‘B’] = new_val # new_val not set in df The warning offers a suggestion to rewrite as … Read more

How to add a new column to an existing DataFrame?

Edit 2017 As indicated in the comments and by @Alexander, currently the best method to add the values of a Series as a new column of a DataFrame could be using assign: df1 = df1.assign(e=pd.Series(np.random.randn(sLength)).values) Edit 2015 Some reported getting the SettingWithCopyWarning with this code. However, the code still runs perfectly with the current pandas … Read more