Share Description
In this course, you will :
- learn how to create and query a SQL table in Spark. Spark SQL adds SQL expressiveness to Spark.
- In this tutorial, you will learn how to use SQL window functions in Spark.
- discovered how to make use of the expressiveness of the SQL window function However, because of this expressiveness, it is now critical that you understand how to properly cache dataframes and SQL tables. It is also critical to understand how to evaluate your application.
- Discover how to do this with the Spark UI. You'll also learn the best way to log in Spark. The query execution plan is another useful tool for tuning query performance issues introduced by Spark SQL.
- Discover how to use the execution plan to assess the provenance of a dataframe.
Syllabus :
- Pyspark SQL
- Using window function sql for natural language processing
- Caching, Logging, and the Spark UI
- Text classification
