ToolBox.File 1.1.2

ToolBox.File - CSV File Reader

A lightweight .NET 9 library for reading and parsing CSV files with built-in FluentValidation support and Result pattern, designed for personal use.

Overview

ToolBox.File is a modern CSV file reader library that enforces data validation through FluentValidation. It provides clean error handling using the Result pattern, eliminating exception-based control flow for business logic while maintaining robust file-level error handling.

Features

• FluentValidation Integration: Enforced validation using FluentValidation for robust data processing
• Result Pattern: Clean error handling without exceptions for business logic
• Multiple Input Sources: Support for files, byte arrays, and streams
• Asynchronous Operations: Full async/await support for non-blocking I/O
• Detailed Validation Results: Per-row validation with comprehensive error reporting
• Flexible CSV Parsing: Support for quoted fields, various line endings (CRLF, LF, CR)
• Type Mapping: Custom mapping functions to convert CSV rows to strongly-typed objects
• Memory Efficient: Streaming parser for large datasets
• UTF-8 Encoding: Automatic UTF-8 encoding with BOM detection

Installation

This is a personal NuGet package. Install via:

dotnet add package ToolBox.File

Or via Package Manager Console:

Install-Package ToolBox.File

Requirements

• .NET 9.0
• FluentValidation 12.0.0+ (automatically included)

Quick Start

1. Define Your Entity and Validator

FluentValidation is required for all CSV operations:

using FluentValidation;

public class Customer
{
    public string Name { get; set; } = string.Empty;
    public int Age { get; set; }
    public string Email { get; set; } = string.Empty;
}

public class CustomerValidator : AbstractValidator<Customer>
{
    public CustomerValidator()
    {
        RuleFor(x => x.Name)
            .NotEmpty()
            .WithMessage("Name is required");
            
        RuleFor(x => x.Age)
            .GreaterThanOrEqualTo(18)
            .WithMessage("Must be at least 18 years old");
            
        RuleFor(x => x.Email)
            .EmailAddress()
            .When(x => !string.IsNullOrEmpty(x.Email))
            .WithMessage("Invalid email format");
    }
}

2. Read and Validate CSV Data

using ToolBox.File;

var validator = new CustomerValidator();

// Map function to convert CSV rows to Customer objects
Customer MapCustomer(string[] row) => new Customer
{
    Name = row[0],
    Age = int.TryParse(row[1], out var age) ? age : 0,
    Email = row.Length > 2 ? row[2] : string.Empty
};

// Read and validate CSV with detailed results
var result = CsvFileReader.ReadAndValidateCsv("customers.csv", MapCustomer, validator);

if (result.IsSuccess)
{
    var summary = result.Value!;
    
    // Process valid customers
    foreach (var customer in summary.ValidEntities)
    {
        Console.WriteLine($"Valid: {customer.Name}, Age: {customer.Age}");
    }
    
    // Handle validation failures
    foreach (var invalidRow in summary.InvalidRows)
    {
        Console.WriteLine($"Row {invalidRow.RowIndex} failed: {invalidRow.ErrorMessage}");
        Console.WriteLine($"Raw data: {string.Join(",", invalidRow.RawFields)}");
    }
    
    Console.WriteLine($"Processed {summary.TotalRows} rows: {summary.ValidCount} valid, {summary.InvalidCount} invalid");
}
else
{
    Console.WriteLine($"Error reading file: {result.Error}");
}

3. Get Only Valid Entities

// Alternative: Get only valid entries, ignore invalid ones
var validResult = CsvFileReader.ReadValidCsv("customers.csv", MapCustomer, validator);

if (validResult.IsSuccess)
{
    var validCustomers = validResult.Value!;
    // Process only valid customers
    validCustomers.ForEach(customer => ProcessCustomer(customer));
}

API Reference

All methods require a FluentValidation IValidator<T> parameter, enforcing data validation.

Primary Methods

ReadAndValidateCsv<T>

Reads a CSV file and returns detailed validation results for all rows.

public static Result<CsvValidationSummary<T>> ReadAndValidateCsv<T>(
    string filePath,
    Func<string[], T> mapFunction,
    IValidator<T> validator,
    bool skipHeader = true,
    bool allowQuotedFields = false)

Parameters:

• filePath: Path to the CSV file
• mapFunction: Function to convert CSV row to entity
• validator: FluentValidation validator for the entity
• skipHeader: Whether to skip the header row (default: true)
• allowQuotedFields: Whether to support quoted CSV fields (default: false)

Returns:

• Result<CsvValidationSummary<T>>: Result containing validation summary or error

ReadValidCsv<T>

Reads a CSV file and returns only entities that pass validation.

public static Result<List<T>> ReadValidCsv<T>(
    string filePath,
    Func<string[], T> mapFunction,
    IValidator<T> validator,
    bool skipHeader = true,
    bool allowQuotedFields = false)

ReadAndValidateCsvBytes<T>

Processes CSV data from a byte array.

public static Result<CsvValidationSummary<T>> ReadAndValidateCsvBytes<T>(
    byte[] content,
    Func<string[], T> mapFunction,
    IValidator<T> validator,
    bool skipHeader = true,
    bool allowQuotedFields = false)

ReadAndValidateCsvStreamAsync<T>

Asynchronously processes CSV data from a stream.

public static async Task<Result<CsvValidationSummary<T>>> ReadAndValidateCsvStreamAsync<T>(
    Stream stream,
    Func<string[], T> mapFunction,
    IValidator<T> validator,
    bool skipHeader = true,
    bool allowQuotedFields = false,
    bool leaveOpen = true)

Result Types

Result<T> / Result

Represents the outcome of an operation:

• IsSuccess: Whether the operation succeeded
• Value: The result value (when successful)
• Error: Error message (when failed)
• Exception: Exception that caused failure (when applicable)

CsvValidationSummary<T>

Contains aggregated validation results:

• ValidEntities: List of entities that passed validation
• InvalidRows: List of rows that failed validation
• TotalRows, ValidCount, InvalidCount: Statistics
• IsValid: Whether all rows passed validation

CsvRowValidationResult<T>

Individual row validation result:

• RowIndex: Zero-based row index
• Entity: The mapped entity
• IsValid: Whether the row passed validation
• RawFields: Original CSV field values
• Failures: FluentValidation failures
• ErrorMessage: Concatenated error messages
• Exception: Exception during mapping (if any)

Usage Examples

Complex Validation Rules

public class ProductValidator : AbstractValidator<Product>
{
    public ProductValidator()
    {
        RuleFor(x => x.Price)
            .GreaterThan(0)
            .WithMessage("Price must be positive");
            
        RuleFor(x => x.Category)
            .Must(BeValidCategory)
            .WithMessage("Invalid product category");
            
        RuleFor(x => x.SKU)
            .Matches(@"^[A-Z]{3}\d{4}$")
            .WithMessage("SKU must follow format: ABC1234");
    }
    
    private bool BeValidCategory(string category)
    {
        var validCategories = new[] { "Electronics", "Clothing", "Books" };
        return validCategories.Contains(category);
    }
}

Processing Large Files with Streaming

public async Task ProcessLargeCsvAsync(Stream csvStream)
{
    var validator = new ProductValidator();
    
    var result = await CsvFileReader.ReadAndValidateCsvStreamAsync(
        csvStream, 
        MapProduct, 
        validator,
        leaveOpen: false);
        
    if (result.IsSuccess)
    {
        var summary = result.Value!;
        
        // Process in batches for memory efficiency
        var batchSize = 1000;
        var validProducts = summary.ValidEntities;
        
        for (int i = 0; i < validProducts.Count; i += batchSize)
        {
            var batch = validProducts.Skip(i).Take(batchSize);
            await ProcessProductBatch(batch);
        }
        
        // Log validation issues
        if (summary.InvalidCount > 0)
        {
            LogValidationIssues(summary.InvalidRows);
        }
    }
}

Batch Processing with Error Reporting

public class CsvProcessor<T>
{
    private readonly IValidator<T> _validator;
    private readonly Func<string[], T> _mapper;
    
    public CsvProcessor(IValidator<T> validator, Func<string[], T> mapper)
    {
        _validator = validator;
        _mapper = mapper;
    }
    
    public async Task<ProcessingResult> ProcessFileAsync(string filePath)
    {
        var result = await CsvFileReader.ReadAndValidateCsvStreamAsync(
            File.OpenRead(filePath), 
            _mapper, 
            _validator,
            leaveOpen: false);
            
        if (result.IsFailure)
        {
            return ProcessingResult.Error(result.Error!);
        }
        
        var summary = result.Value!;
        
        // Save valid entities
        await SaveValidEntities(summary.ValidEntities);
        
        // Generate error report
        if (summary.InvalidCount > 0)
        {
            await GenerateErrorReport(summary.InvalidRows, filePath);
        }
        
        return ProcessingResult.Success(summary.ValidCount, summary.InvalidCount);
    }
}

Error Handling

The library uses a Result pattern for clean error handling:

File-Level Errors

• File not found, permission issues, I/O errors
• Returned as Result.Failure with error message and exception

Row-Level Errors

• Mapping failures (e.g., invalid data types)
• Validation failures (FluentValidation rules)
• Captured in CsvRowValidationResult objects

No Exception-Based Control Flow

Business logic errors (validation failures, data issues) are returned as Result objects, not thrown as exceptions.

var result = CsvFileReader.ReadValidCsv(filePath, mapper, validator);

// No try-catch needed for business logic errors
if (result.IsSuccess)
{
    ProcessData(result.Value!);
}
else
{
    LogError(result.Error!);
}

CSV Format Support

• Delimiters: Comma-separated values
• Line Endings: CRLF (\r\n), LF (\n), CR (\r)
• Quoted Fields: Optional support for quoted fields with escaped quotes ("")
• Headers: Optional header row skipping
• Empty Lines: Automatically skipped
• Encoding: UTF-8 with automatic BOM detection

Performance Considerations

1. Memory Usage: Large CSV files are loaded into memory. For very large files (>1GB), consider processing in chunks or using streaming approaches.

2. Validation Cost: FluentValidation adds overhead. Complex rules may impact performance on large datasets.

3. Allocation: Result objects and validation results create allocations. Consider object pooling for high-throughput scenarios.

4. Async Operations: Use async methods for large files to avoid blocking threads.

Best Practices

1. Robust Mapping Functions

Handle missing or invalid data gracefully:

Customer MapCustomer(string[] row) => new Customer
{
    Name = row.Length > 0 ? row[0] : string.Empty,
    Age = row.Length > 1 && int.TryParse(row[1], out var age) ? age : 0,
    Email = row.Length > 2 ? row[2] : string.Empty
};

2. Comprehensive Validation

Use FluentValidation's rich feature set:

public class CustomerValidator : AbstractValidator<Customer>
{
    public CustomerValidator()
    {
        RuleFor(x => x.Name)
            .NotEmpty()
            .Length(2, 100)
            .WithMessage("Name must be between 2 and 100 characters");
            
        RuleFor(x => x.Age)
            .InclusiveBetween(0, 120)
            .WithMessage("Age must be between 0 and 120");
            
        RuleFor(x => x.Email)
            .EmailAddress()
            .When(x => !string.IsNullOrEmpty(x.Email));
    }
}

3. Error Reporting

Generate meaningful error reports:

private void GenerateErrorReport(IEnumerable<CsvRowValidationResult<Customer>> invalidRows)
{
    var report = invalidRows.Select(row => new
    {
        RowIndex = row.RowIndex + 1, // 1-based for user display
        RawData = string.Join(",", row.RawFields),
        Errors = row.ErrorMessage
    });
    
    // Save to file, send email, etc.
}

Limitations

1. Custom Delimiters: Only comma (,) is supported as a field delimiter
2. Memory Usage: Files are loaded entirely into memory
3. Complex CSV: Limited support for advanced CSV features like multi-line fields

Troubleshooting

Common Issues

Problem: Result.IsFailure due to file access Solution: Check file permissions and path validity

Problem: All rows marked as invalid Solution: Verify your mapping function handles the expected CSV structure and FluentValidation rules are appropriate

Problem: Performance issues with large files Solution: Use async methods and consider processing valid entities in batches

Version History

• 1.1.1: Current version with FluentValidation integration and Result pattern

License

MIT License - This project is intended for personal usage.

Author

Guillaume Bodson

This library is designed for personal use and enforces best practices through FluentValidation integration.