Scala - Read/Write File (Source.fromFile)
• Scala’s Source.fromFile provides a simple API for reading text files with automatic resource management through try-with-resources patterns or using Using from Scala 2.13+
Readwrite
Read-Write Lock: Concurrent Readers, Exclusive Writers
Standard mutexes are blunt instruments. When you lock a mutex to read shared data, you block every other thread—even those that only want to read. This is wasteful. Reading doesn’t modify state, so…
Read more →R - Read/Write RDS and RData Files
R provides two native binary formats for persisting objects: RDS and RData. RDS files store a single R object, while RData files can store multiple objects from your workspace. Both formats preserve…
Read more →Python - Read/Write Binary Files
Binary files contain raw bytes without text encoding interpretation. Unlike text files, binary mode preserves exact byte sequences, making it critical for non-text data.
Read more →Python - Read/Write CSV Files
The csv module provides straightforward methods for reading CSV files. The csv.reader() function returns an iterator that yields each row as a list of strings.
Python - Read/Write Excel Files (openpyxl/xlsxwriter)
pip install openpyxl xlsxwriter pandas
Read more →Python - Read/Write JSON Files
• Python’s json module provides load()/loads() for reading and dump()/dumps() for writing JSON data with built-in type conversion between Python objects and JSON format
Go sync.RWMutex: Read-Write Locks
Most concurrent data structures face a common challenge: reads vastly outnumber writes. Think about a configuration store that’s read thousands of times per second but updated once per hour, or a…
Read more →Apache Spark - Read/Write from HDFS
Spark reads from and writes to HDFS through Hadoop’s FileSystem API. When running on a Hadoop cluster with YARN or Mesos, Spark automatically detects HDFS configuration from core-site.xml and…
Apache Spark - Read/Write from S3
Spark uses the Hadoop S3A filesystem implementation to interact with S3. You need the correct dependencies and AWS credentials configured before reading or writing data.
Read more →Apache Spark - Read/Write JDBC Databases
Before reading or writing data, ensure the appropriate JDBC driver is available to all Spark executors. For cluster deployments, include the driver JAR using --jars or --packages:
Apache Spark - Read/Write from Azure Blob/ADLS
Apache Spark requires specific libraries to communicate with Azure storage. Add these dependencies to your pom.xml for Maven projects:
Apache Spark - Read/Write from GCS
Apache Spark doesn’t include GCS support out of the box. You need the Cloud Storage connector JAR that implements the Hadoop FileSystem interface for gs:// URIs.