query(self, expr: 'str', inplace: 'bool' = False, **kwargs)
The result of the evaluation of this expression is first passed to DataFrame.loc
and if that fails because of a multidimensional key (e.g., a DataFrame) then the result will be passed to DataFrame.__getitem__
.
This method uses the top-level eval
function to evaluate the passed query.
The ~pandas.DataFrame.query
method uses a slightly modified Python syntax by default. For example, the &
and |
(bitwise) operators have the precedence of their boolean cousins, and
and or
. This is syntactically valid Python, however the semantics are different.
You can change the semantics of the expression by passing the keyword argument parser='python'
. This enforces the same semantics as evaluation in Python space. Likewise, you can pass engine='python'
to evaluate an expression using Python itself as a backend. This is not recommended as it is inefficient compared to using numexpr
as the engine.
The DataFrame.index
and DataFrame.columns
attributes of the ~pandas.DataFrame
instance are placed in the query namespace by default, which allows you to treat both the index and columns of the frame as a column in the frame. The identifier index
is used for the frame index; you can also use the name of the index to identify it in a query. Please note that Python keywords may not be used as identifiers.
For further details and examples see the query
documentation in indexing <indexing.query>
.
Backtick quoted variables
Backtick quoted variables are parsed as literal Python code and are converted internally to a Python valid identifier. This can lead to the following problems.
During parsing a number of disallowed characters inside the backtick quoted string are replaced by strings that are allowed as a Python identifier. These characters include all operators in Python, the space character, the question mark, the exclamation mark, the dollar sign, and the euro sign. For other characters that fall outside the ASCII range (U+0001..U+007F) and those that are not further specified in PEP 3131, the query parser will raise an error. This excludes whitespace different than the space character, but also the hashtag (as it is used for comments) and the backtick itself (backtick can also not be escaped).
In a special case, quotes that make a pair around a backtick can confuse the parser. For example, `it's` > `that's`
will raise an error, as it forms a quoted string ( 's > `that'
) with a backtick inside.
See also the Python documentation about lexical analysis (https://docs.python.org/3/reference/lexical_analysis.html) in combination with the source code in pandas.core.computation.parsing
.
The query string to evaluate.
You can refer to variables in the environment by prefixing them with an '@' character like @a + b
.
You can refer to column names that are not valid Python variable names by surrounding them in backticks. Thus, column names containing spaces or punctuations (besides underscores) or starting with digits must be surrounded by backticks. (For example, a column named "Area (cm^2)" would be referenced as `Area (cm^2)`
). Column names which are Python keywords (like "list", "for", "import", etc) cannot be used.
For example, if one of your columns is called a a
and you want to sum it with b
, your query should be `a a` + b
.
Backtick quoting introduced.
Expanding functionality of backtick quoting for more than only spaces.
Whether the query should modify the data in place or return a modified copy.
See the documentation for eval
for complete details on the keyword arguments accepted by DataFrame.query
.
DataFrame resulting from the provided query expression or None if inplace=True
.
Query the columns of a DataFrame with a boolean expression.
DataFrame.eval
Evaluate a string describing operations on DataFrame columns.
eval
Evaluate a string describing operations on DataFrame columns.
>>> df = pd.DataFrame({'A': range(1, 6),This example is valid syntax, but we were not able to check execution
... 'B': range(10, 0, -2),
... 'C C': range(10, 5, -1)})
... df A B C C 0 1 10 10 1 2 8 9 2 3 6 8 3 4 4 7 4 5 2 6
>>> df.query('A > B') A B C C 4 5 2 6
The previous expression is equivalent to
This example is valid syntax, but we were not able to check execution>>> df[df.A > df.B] A B C C 4 5 2 6
For columns with spaces in their name, you can use backtick quoting.
This example is valid syntax, but we were not able to check execution>>> df.query('B == `C C`') A B C C 0 1 10 10
The previous expression is equivalent to
This example is valid syntax, but we were not able to check execution>>> df[df.B == df['C C']] A B C C 0 1 10 10See :
The following pages refer to to this document either explicitly or contain code examples using this.
pandas.core.frame.DataFrame.eval
pandas.core.computation.eval.eval
Hover to see nodes names; edges to Self not shown, Caped at 50 nodes.
Using a canvas is more power efficient and can get hundred of nodes ; but does not allow hyperlinks; , arrows or text (beyond on hover)
SVG is more flexible but power hungry; and does not scale well to 50 + nodes.
All aboves nodes referred to, (or are referred from) current nodes; Edges from Self to other have been omitted (or all nodes would be connected to the central node "self" which is not useful). Nodes are colored by the library they belong to, and scaled with the number of references pointing them