Indexing in a database is a technique that improves the speed of data retrieval operations. It works by creating a data structure known as an index, which allows for faster searching and accessing of data from the database.
An index is a small table with two columns. The first column contains a copy of the primary or candidate key of a table, and the second column contains a set of pointers, each of which holds the address of the disk block where the specific key value is stored
When a query needs to access a table that has an index, the database can use the index to retrieve records faster. The database finds the key value in the index, and each key value is stored with a pointer to the record in the table associated with this key value.
Indexes are not required for a database to function, but they can significantly improve query response time. However, they do require additional space to store the index records.
There are different types of indexing methods such as Primary Indexing and Secondary Indexing. In Primary Indexing, there is a one-to-one relationship between entries in the index table. This type of indexing can be further divided into Dense Index, where a record is created for every search key value in the database, and Sparse Index, where an index record appears only for some of the values in the file.
Secondary Indexing can be generated by a field which has a unique value for each record, and it should be a candidate key. This type of indexing is used to reduce the mapping size of the first level.
In summary, indexing is a powerful technique that can greatly enhance the performance of a database by enabling faster retrieval of data.
So the we can see @Index annotations are used to define database indexes for CustomerEntity. Then in every @Index annotation we define some parameter as name, columnList and unique. For example @Index(name = "uniqueEmailIndex", columnList = "email", unique = true)
name: we define the name of the index.
columnList: we specify the column of the table that we will create an index.
unique: specifies that the values in the column must be unique, meaning that no two rows in the table can have the same value. This enforces data integrity and prevents duplicate in the table.
We can also create a composite index for multi column in a table by defining columns name in the columnList parameter. Then in this case if we define the unique = true then it means no two rows in the table can have both the same email and phone values simultaneously.
Note: In JPA, if we use annotation @Column(name = "email", unique = true) then JPA also will create for us an index for that column automatically. However, the index name will have a random name.
<plugins><plugin><groupId>org.openapitools</groupId><artifactId>openapi-generator-maven-plugin</artifactId><version>7.0.1</version><executions><execution><goals><goal>generate</goal></goals><configuration><!-- path to the openapi file spec `.yml` --><inputSpec>${project.basedir}/src/main/resources/openapi/openapi-server.yml
</inputSpec><generatorName>spring</generatorName><!-- generated package for api interface --><apiPackage>com.springboot.project.generated.api</apiPackage><!-- generated package for models --><modelPackage>com.springboot.project.generated.model</modelPackage><!-- using supportingFilesToGenerate --><supportingFilesToGenerate>ApiUtil.java
</supportingFilesToGenerate><configOptions><useTags>true</useTags><delegatePattern>true</delegatePattern><dateLibrary>java8</dateLibrary><java8>false</java8><interfaceOnly>true</interfaceOnly><useBeanValidation>true</useBeanValidation><performBeanValidation>true</performBeanValidation><useOptional>false</useOptional><useSpringBoot3>true</useSpringBoot3></configOptions></configuration></execution></executions></plugin><plugin><groupId>org.apache.maven.plugins</groupId><artifactId>maven-compiler-plugin</artifactId><version>3.11.0</version><configuration><source>17</source><target>17</target><annotationProcessorPaths><path><groupId>org.mapstruct</groupId><artifactId>mapstruct-processor</artifactId><version>1.5.5.Final</version></path><path><groupId>org.projectlombok</groupId><artifactId>lombok</artifactId><version>1.18.28</version></path><path><groupId>org.projectlombok</groupId><artifactId>lombok-mapstruct-binding</artifactId><version>0.2.0</version></path></annotationProcessorPaths></configuration></plugin></plugins>
Now, in the folder resource, we will create a folder openapi and a openapi-server.yaml inside it. The content of this file will look like below with some apis.
openapi:3.0.3info:title:Swagger Customer - OpenAPI 3.0description:Everything about sample hibernate second level cachetermsOfService:http://swagger.io/terms/contact:email:apiteam@swagger.iolicense:name:Apache 2.0url:http://www.apache.org/licenses/LICENSE-2.0.htmlversion:1.0.11externalDocs:description:Find out more about Swaggerurl:http://swagger.ioservers:-url:https://petstore3.swagger.io/api/v1tags:-name:customerdescription:Everything about your CustomerexternalDocs:description:Find out moreurl:http://swagger.io-name:orderdescription:Everything about your OrderexternalDocs:description:Find out moreurl:http://swagger.iopaths:/v1/customers:get:tags:-customersummary:get an existing customer infodescription:get an existing customer info by EmailoperationId:getCustomerInfoByEmailparameters:-name:emailin:queryrequired:trueschema:type:stringresponses:'200':description:Successful operationcontent:application/json:schema:$ref:'#/components/schemas/CustomerResponse''400':description:Invalid ID supplied'404':description:Pet not found'405':description:Validation exceptionpost:tags:-customersummary:Add a Customer to databasedescription:Add a Customer to databaseoperationId:addCustomerrequestBody:description:Create a Customer to databasecontent:application/json:schema:$ref:'#/components/schemas/CustomerRequest'required:trueresponses:'200':description:Successful operationcontent:application/json:schema:$ref:'#/components/schemas/CustomerResponse''405':description:Invalid input/v1/customers/{customerId}/info:get:tags:-customersummary:get an existing customer infodescription:get an existing customer info by IdoperationId:getCustomerInfoparameters:-name:customerIdin:pathrequired:trueschema:type:stringformat:uuidresponses:'200':description:Successful operationcontent:application/json:schema:$ref:'#/components/schemas/CustomerResponse''400':description:Invalid ID supplied'404':description:Pet not found'405':description:Validation exceptionput:tags:-customersummary:Update an existing customerdescription:Update an existing customer by IdoperationId:updateCustomersparameters:-name:customerIdin:pathrequired:trueschema:type:stringformat:uuidrequestBody:description:Update an existent customer in the databasecontent:application/json:schema:$ref:'#/components/schemas/CustomerResponse'required:trueresponses:'200':description:Successful operationcontent:application/json:schema:$ref:'#/components/schemas/CustomerResponse''400':description:Invalid ID supplied'404':description:Pet not found'405':description:Validation exception/v1/customers/{customerId}/orders:post:tags:-ordersummary:Add an order to databasedescription:Add an order to databaseoperationId:addOrderparameters:-name:customerIdin:pathrequired:trueschema:type:stringformat:uuidrequestBody:description:Create an Order to databasecontent:application/json:schema:$ref:'#/components/schemas/OrderRequest'required:trueresponses:'200':description:Successful operationcontent:application/json:schema:$ref:'#/components/schemas/OrderResponse''405':description:Invalid input/v1/customers/{customerId}/orders/{orderId}:get:tags:-ordersummary:Get an order from databasedescription:Get an order from databaseoperationId:getOrderDetailparameters:-name:customerIdin:pathrequired:trueschema:type:stringformat:uuid-name:orderIdin:pathrequired:trueschema:type:stringformat:uuidresponses:'200':description:Successful operationcontent:application/json:schema:$ref:'#/components/schemas/OrderResponse''405':description:Invalid inputcomponents:schemas:CustomerResponse:allOf:-$ref:'#/components/schemas/CustomerRequest'-type:objectproperties:id:type:stringformat:uuidOrderResponse:allOf:-$ref:'#/components/schemas/OrderRequest'-type:objectproperties:id:type:stringformat:uuidItemResponse:allOf:-$ref:'#/components/schemas/ItemRequest'-type:objectproperties:id:type:stringformat:uuidCustomerRequest:type:objectrequired:-email-phoneproperties:fullName:type:stringemail:type:stringaddress:type:stringphone:type:stringgender:type:stringenum:-M-Fdob:type:stringformat:dateOrderRequest:type:objectrequired:-orderName-orderStatusproperties:orderName:type:stringorderStatus:type:stringenum:-CREATED-SHIPPING-FINISHEDitems:type:arrayitems:$ref:'#/components/schemas/ItemRequest'ItemRequest:type:objectrequired:-itemName-quantity-priceproperties:itemName:type:stringquantity:type:integerformat:int64price:type:numberformat:double
Now, we can use command mvn clean install to build and generate apis and model from openapi-server.yaml.
Then let's define Entities as below. In this example a CustomerEnitty will contain many OrderEntity, then in an OrderEntity will contain many ItemEntity.
In CustomerEntity we will put index for email, phone and a composite index for email and phone.
packagecom.springboot.project.service;importcom.springboot.project.entity.CustomerEntity;importcom.springboot.project.mapper.AutoCustomerMapper;importcom.springboot.project.model.Customer;importcom.springboot.project.repository.CustomerRepository;importlombok.AllArgsConstructor;importorg.springframework.beans.factory.annotation.Autowired;importorg.springframework.stereotype.Service;importjava.util.Optional;importjava.util.UUID;@Service@AllArgsConstructor(onConstructor=@__(@Autowired))publicclassCustomerService{privatefinalCustomerRepositorycustomerRepository;publicCustomercreateCustomer(Customercustomer){CustomerEntitycustomerEntity=AutoCustomerMapper.MAPPER.mapToCustomerEntity(customer);customerEntity=this.customerRepository.save(customerEntity);returnAutoCustomerMapper.MAPPER.mapToCustomer(customerEntity);}publicCustomergetCustomer(UUIDcustomerId){Optional<CustomerEntity>customerEntity=this.customerRepository.findById(customerId);if(customerEntity.isPresent()){returnAutoCustomerMapper.MAPPER.mapToCustomer(customerEntity.get());}thrownewRuntimeException("Customer Not Found!");}publicvoidupdateCustomer(UUIDcustomerId,Customercustomer){Optional<CustomerEntity>customerEntity=this.customerRepository.findById(customerId);if(customerEntity.isPresent()){CustomerEntityexistedCustomerEntity=customerEntity.get();CustomerEntityupdateCustomerEntity=AutoCustomerMapper.MAPPER.mapToCustomerEntity(customer);AutoCustomerMapper.MAPPER.updateCustomerEntity(existedCustomerEntity,updateCustomerEntity);this.customerRepository.save(existedCustomerEntity);return;}thrownewRuntimeException("Customer Not Found!");}publicvoiddeleteCustomer(UUIDcustomerId){this.customerRepository.deleteById(customerId);}publicCustomerfindCustomerByEmail(Stringemail){Optional<CustomerEntity>customerEntity=this.customerRepository.findCustomerByEmail(email);if(customerEntity.isPresent()){returnAutoCustomerMapper.MAPPER.mapToCustomer(customerEntity.get());}thrownewRuntimeException("Customer Not Found! with email: "+email);}}
packagecom.springboot.project.service;importcom.springboot.project.entity.CustomerEntity;importcom.springboot.project.entity.ItemEntity;importcom.springboot.project.entity.OrderEntity;importcom.springboot.project.mapper.AutoItemMapper;importcom.springboot.project.mapper.AutoOrderMapper;importcom.springboot.project.model.Item;importcom.springboot.project.model.Order;importcom.springboot.project.repository.CustomerRepository;importcom.springboot.project.repository.OrderRepository;importlombok.AllArgsConstructor;importorg.springframework.beans.factory.annotation.Autowired;importorg.springframework.stereotype.Service;importorg.springframework.util.CollectionUtils;importjava.util.List;importjava.util.Optional;importjava.util.UUID;importjava.util.stream.Collectors;@Service@AllArgsConstructor(onConstructor=@__(@Autowired))publicclassOrderService{privatefinalCustomerRepositorycustomerRepository;privatefinalOrderRepositoryorderRepository;publicOrdercreateOrder(UUIDcustomerId,Orderorder){if(CollectionUtils.isEmpty(order.getItems())){thrownewRuntimeException("Can not create Order without any Item!");}CustomerEntitycustomerEntity=this.getCustomer(customerId);OrderEntityorderEntity=AutoOrderMapper.MAPPER.mapToEntity(order);List<ItemEntity>itemEntities=this.mapToItemEntities(order.getItems(),orderEntity);orderEntity.setCustomer(customerEntity);orderEntity.setItems(itemEntities);orderEntity=this.orderRepository.save(orderEntity);returnAutoOrderMapper.MAPPER.mapToOrderFromEntity(orderEntity);}publicOrdergetOrderDetail(UUIDcustomerId,UUIDorderId){Optional<OrderEntity>orderEntityOpt=this.orderRepository.findById(orderId);if(orderEntityOpt.isPresent()){OrderEntityorderEntity=orderEntityOpt.get();if(orderEntity.getCustomer().getId().equals(customerId)){returnAutoOrderMapper.MAPPER.mapToOrderFromEntity(orderEntity);}}thrownewRuntimeException("customerId or orderId is not correct or relative!");}privateCustomerEntitygetCustomer(UUIDcustomerId){Optional<CustomerEntity>customerEntity=this.customerRepository.findById(customerId);if(customerEntity.isPresent()){returncustomerEntity.get();}thrownewRuntimeException("Customer Not Found!");}privateList<ItemEntity>mapToItemEntities(List<Item>items,OrderEntityorderEntity){returnitems.stream().map(AutoItemMapper.MAPPER::toItemEntity).peek(i->i.setOrder(orderEntity)).collect(Collectors.toList());}}
packagecom.springboot.project.service;importcom.springboot.project.entity.ItemEntity;importcom.springboot.project.entity.OrderEntity;importcom.springboot.project.mapper.AutoItemMapper;importcom.springboot.project.model.Item;importcom.springboot.project.repository.CustomerRepository;importcom.springboot.project.repository.ItemRepository;importcom.springboot.project.repository.OrderRepository;importlombok.AllArgsConstructor;importorg.springframework.beans.factory.annotation.Autowired;importorg.springframework.stereotype.Service;importjava.util.List;importjava.util.Optional;importjava.util.UUID;importjava.util.stream.Collectors;@Service@AllArgsConstructor(onConstructor=@__(@Autowired))publicclassItemService{privatefinalItemRepositoryitemRepository;privatefinalCustomerRepositorycustomerRepository;privatefinalOrderRepositoryorderRepository;privateList<Item>addItemToOrder(UUIDorderId,List<Item>items){OrderEntityorderEntity=this.getOrderEntity(orderId);List<ItemEntity>itemEntities=this.toItemEntities(items);itemEntities.forEach(i->i.setOrder(orderEntity));List<ItemEntity>itemEntityResults=this.itemRepository.saveAll(itemEntities);returnthis.toItems(itemEntityResults);}privateList<ItemEntity>toItemEntities(List<Item>items){returnitems.stream().map(AutoItemMapper.MAPPER::toItemEntity).collect(Collectors.toList());}privateList<Item>toItems(List<ItemEntity>itemEntities){returnitemEntities.stream().map(AutoItemMapper.MAPPER::toItemFromEntity).collect(Collectors.toList());}privateOrderEntitygetOrderEntity(UUIDorderId){Optional<OrderEntity>orderEntity=this.orderRepository.findById(orderId);if(orderEntity.isPresent()){returnorderEntity.get();}thrownewRuntimeException("Order Not Found!");}}
Now, let's start the application and use a Database tool to check the tables.
As we can see, Indexes in customers table are created with the name and unique is true as we configured in the CustomerEntity.
Next, we can also see the index of column item_name is created when we use @Column(name = "item_name", unique = true) but the Index Name is a random name.
Now, we will test how do the indexing help us in querying in database.
Firstly, let's use the postman and with the request body as below.
Index Scan using uniquemultiindex on customers c (cost=0.29..8.30 rows=1 width=97) (actual time=0.016..0.017 rows=1 loops=1)
Index Cond: ((email)::text = 'Greyson_McClure41@example.net'::text)
Planning Time: 0.054 ms
Execution Time: 0.029 ms
As you can see, finding a record in 10000 rows just took 0.029ms, it is so fast.
Now, let's use the sql command below to to analyze the select query using no indexed column full_name as below.
We have just known about what is index and how to apply indexing with Jpa.
We also do some tests with indexing in database and see the different between indexed and no indexed.
However, we also need to consider applying indexing:
Pros: Indexing improve performance for querying in the database.
Cons: Indexing will take more space of the table in database and it also can slow down the INSERT statements in database because:
When we INSERT new row, we are not also updating the table but also the indexes in that table.
Most indexes use some form of a tree data structure. When a new entry is inserted, this tree needs to be rebalanced. Depending on the type of the index and the distribution of data, The rebalancing operation can be costly.
If there are concurrent transactions happening on the same table, having an index can increase the chances of lock contention, which can also slow down INSERT operations.