The Databricks SQL Connector supports connecting through HTTP and HTTPS proxy servers with various authentication methods. This feature automatically detects system proxy configuration and handles proxy authentication transparently.

The connector automatically uses your system's proxy configuration when available:

from databricks import sql

# Basic connection - uses system proxy automatically
with sql.connect(
    server_hostname="your-workspace.cloud.databricks.com",
    http_path="/sql/1.0/endpoints/your-endpoint-id",
    access_token="your-token"
) as connection:
    # Your queries here...

For advanced proxy authentication (like Kerberos), specify the authentication method:

with sql.connect(
    server_hostname="your-workspace.cloud.databricks.com",
    http_path="/sql/1.0/endpoints/your-endpoint-id", 
    access_token="your-token",
    _proxy_auth_method="negotiate"  # Enable Kerberos proxy auth
) as connection:
    # Your queries here...

The connector follows standard proxy environment variable conventions:

Note: The connector also recognizes lowercase versions (http_proxy, https_proxy, no_proxy).

Basic proxy (no authentication):

export HTTPS_PROXY="http://proxy.company.com:8080"

Proxy with basic authentication:

export HTTPS_PROXY="http://username:password@proxy.company.com:8080"

The connector supports multiple proxy authentication methods via the _proxy_auth_method parameter:

Default behavior when credentials are provided in the proxy URL or when _proxy_auth_method="basic" is specified.

# Method 1: Credentials in proxy URL (recommended)
# Set environment: HTTPS_PROXY="http://user:pass@proxy.company.com:8080"
with sql.connect(
    server_hostname="your-workspace.com",
    http_path="/sql/1.0/endpoints/abc123",
    access_token="your-token"
    # No _proxy_auth_method needed - detected automatically
) as conn:
    pass

# Method 2: Explicit basic authentication
with sql.connect(
    server_hostname="your-workspace.com", 
    http_path="/sql/1.0/endpoints/abc123",
    access_token="your-token",
    _proxy_auth_method="basic"  # Explicit basic auth
) as conn:
    pass

For corporate environments using Kerberos authentication with proxy servers.

Prerequisites:

• Valid Kerberos tickets (run kinit first)
• Properly configured Kerberos environment

with sql.connect(
    server_hostname="your-workspace.com",
    http_path="/sql/1.0/endpoints/abc123", 
    access_token="your-token",
    _proxy_auth_method="negotiate"  # Enable Kerberos proxy auth
) as conn:
    pass

Kerberos Setup Example:

# Obtain Kerberos tickets
kinit your-username@YOUR-DOMAIN.COM

# Set proxy (no credentials in URL for Kerberos)
export HTTPS_PROXY="http://proxy.company.com:8080"

# Run your Python script
python your_script.py

The connector respects system proxy bypass rules. Requests to hosts listed in NO_PROXY or system bypass lists will connect directly, bypassing the proxy.

# Bypass proxy for local and internal hosts
export NO_PROXY="localhost,127.0.0.1,*.internal.company.com,10.*"

The connector automatically makes per-request decisions about proxy usage based on:

1. System proxy configuration - Detected from environment variables
2. Proxy bypass rules - Honor NO_PROXY and system bypass settings
3. Target host - Check if the specific host should use proxy

The connector maintains separate connection pools for direct and proxy connections, allowing efficient handling of mixed proxy/direct traffic.

HTTPS connections through HTTP proxies use the CONNECT method for SSL tunneling. The connector handles this automatically while preserving all SSL verification settings.

Problem: Connection fails with proxy-related errors

Solution: 
1. Verify proxy environment variables are set correctly
2. Check if proxy requires authentication
3. Ensure proxy allows CONNECT method for HTTPS
4. Test proxy connectivity with curl:
   curl -x $HTTPS_PROXY https://your-workspace.com

Problem: Kerberos authentication fails

Solution:
1. Verify Kerberos tickets: klist
2. Renew tickets if expired: kinit
3. Check proxy supports negotiate authentication
4. Ensure time synchronization between client and KDC

Problem: Some requests bypass proxy unexpectedly

Solution:
1. Check NO_PROXY environment variable
2. Review system proxy bypass settings
3. Verify the target hostname format

Enable detailed logging to troubleshoot proxy issues:

import logging

# Enable connector debug logging
logging.basicConfig(level=logging.DEBUG)
logging.getLogger("databricks.sql").setLevel(logging.DEBUG)

# Enable urllib3 logging for HTTP details
logging.getLogger("urllib3").setLevel(logging.DEBUG)

Use the provided example script to test different proxy authentication methods:

cd examples/
python proxy_authentication.py

This script tests:

• Default proxy behavior
• Basic authentication
• Kerberos/Negotiate authentication

See examples/proxy_authentication.py for a comprehensive demonstration of proxy authentication methods.

1. Environment Variables: Check HTTP_PROXY/HTTPS_PROXY environment variables
2. System Configuration: Use Python's urllib.request.getproxies() to detect system settings
3. Bypass Rules: Honor NO_PROXY and urllib.request.proxy_bypass() rules
4. Per-Request Logic: Decide proxy usage for each request based on target host

• HTTP Proxies: For both HTTP and HTTPS traffic (via CONNECT)
• HTTPS Proxies: Encrypted proxy connections
• Authentication: Basic, Negotiate/Kerberos
• Bypass Rules: Full support for NO_PROXY patterns

The connector uses a unified HTTP client that maintains:

• Direct Pool Manager: For non-proxy connections
• Proxy Pool Manager: For proxy connections
• Per-Request Routing: Automatic selection based on target host

This architecture ensures optimal performance and correct proxy handling across all connector operations.