The smallest individual unit of the program is known as a token or a lexical unit.
Python has the following tokens:
-
Keywords
-
Identifiers
-
Literals
-
Operators
-
Punctuators
-
Keywords:
A keyword is a word having a special meaning reserved by programming language.
Examples:- false, assert, del, for, in, or, while, none, break, elif, from, is, pass, with, true, class, else, global, lambda, raise, yield, and, continue, except, if, nonlocal, return, as, def, finally, import, not, try
-
Identifiers:
Identifiers are the names given to the different parts of the program viz. variable, objects, classes, functions, lists, dictionaries, and so forth.
-
The naming rules of python identifier can be summarized as follow:
-
Variable name must only be a non-keyword word with no spaces in between.
-
Variable names must be only letters, numbers, and underscore (_).
-
Variables’ names cannot begin with a number, although they can contain numbers.
Literals/values:
Literals are data items that have a fixed/constant value.
Python allows several kinds of literals, which are being given below.
String literals:
A string literal is a sequence of characters surroundings by quotes
String literal can either be a single-line string or multi-line string
Numeric literal:
Numeric literals are numeric values it can be:
-
Int (sign integer) it called integers or int
-
Floating
-
Complex number literals
Boolean literal:
It used to represent the Boolean value (true/false)
Operators
It is an action applied on variables and in other objects in an expression
Types of operators:
Arithmetic operators- (+, -, *, /, %, **, //)
Bitwise operators- (^, &, |)
Shift operators- (<<,>>)
Identify operators- (is, is not)
Relational operators- (>, <, >=, <=, ==, =!)
Logical operator- (and, or)
Assignment operator- (=)
Membership operators- (in, not in)
Arithmetic assignment operators- (/=, +=, -=, */, %=, **=, //=)
Punctuators:
Symbols that are used in programing language to organize sentence structure
Ex= ‘“ # ( ) [ ] { } @ , : . ` =
