Skip to main content
Search

# cuDF skills for Apache Spark

1 related articles

Subscribe to updates
  1. Spark UDFs no longer stall the GPU: agents cut rewrites to hours, one job runs 22x faster

    NVIDIA released a set of skills for coding agents that rewrite Apache Spark user-defined functions (UDFs) into GPU-accelerable implementations for the cuDF plugin. UDFs previously broke Spark's declarative execution, forcing the accelerator to convert columnar data to row-wise form and fall back to the CPU, paying a data round trip and forgoing GPU compute. The skills split the rewrite into four verification-driven loops for testing, conversion, benchmarking and optimization, and supply a project template plus cuDF programming guidance; a major consumer technology company used the workflow to convert 30+ production UDFs, with one UDF affecting more than 3,000 jobs showing a 22x speedup in that job, cutting rewrite iteration from days or weeks to hours.

You’re all caught up