|
| 1 | +# Build Guide |
| 2 | + |
| 3 | +This guide provides comprehensive instructions for building and developing the Microsoft Python Driver for SQL Server (`mssql-python`) from source. It covers all supported platforms and explains how to build the native pybind bindings, create Python wheels, and run tests. |
| 4 | + |
| 5 | +## Prerequisites |
| 6 | + |
| 7 | +### Python Requirements |
| 8 | +- **Python 3.10 or higher** is required |
| 9 | +- Use the standard `python` and `pip` commands (no Anaconda or pyenv dependencies) |
| 10 | + |
| 11 | +### General Dependencies |
| 12 | +Install the required Python packages: |
| 13 | +```bash |
| 14 | +pip install -r requirements.txt |
| 15 | +``` |
| 16 | + |
| 17 | +This will install: |
| 18 | +- `pytest` - Testing framework |
| 19 | +- `pytest-cov` - Test coverage reporting |
| 20 | +- `pybind11` - Python/C++ binding library |
| 21 | +- `coverage` - Code coverage measurement |
| 22 | +- `unittest-xml-reporting` - XML test reporting |
| 23 | +- `setuptools` - Package building tools |
| 24 | + |
| 25 | +## Platform-Specific Prerequisites |
| 26 | + |
| 27 | +### Windows |
| 28 | + |
| 29 | +#### Required Software |
| 30 | +1. **Visual Studio Build Tools 2022** or **Visual Studio 2022** with C++ support |
| 31 | + - Download from: https://visualstudio.microsoft.com/downloads/ |
| 32 | + - During installation, select the **"Desktop development with C++"** workload |
| 33 | + - This automatically includes CMake |
| 34 | + |
| 35 | +2. **PyBind11**: |
| 36 | + ```cmd |
| 37 | + pip install pybind11 |
| 38 | + ``` |
| 39 | + |
| 40 | +#### Architecture Support |
| 41 | +- x64 (64-bit Intel/AMD) |
| 42 | +- x86 (32-bit Intel/AMD) |
| 43 | +- ARM64 (64-bit ARM) |
| 44 | + |
| 45 | +### macOS |
| 46 | + |
| 47 | +#### Required Software |
| 48 | +1. **Xcode Command Line Tools**: |
| 49 | + ```bash |
| 50 | + xcode-select --install |
| 51 | + ``` |
| 52 | + |
| 53 | +2. **CMake and PyBind11**: |
| 54 | + ```bash |
| 55 | + brew install cmake |
| 56 | + pip install pybind11 |
| 57 | + ``` |
| 58 | + |
| 59 | +3. **Microsoft ODBC Driver 18**: |
| 60 | + ```bash |
| 61 | + brew tap microsoft/mssql-release https://github.com/Microsoft/homebrew-mssql-release |
| 62 | + ACCEPT_EULA=Y brew install msodbcsql18 |
| 63 | + ``` |
| 64 | + |
| 65 | + > **Note**: This provides development headers (`sql.h`, `sqlext.h`) and the dynamic library (`libmsodbcsql.18.dylib`) required for building native bindings. |
| 66 | +
|
| 67 | +#### Architecture Support |
| 68 | +- Universal2 binaries (ARM64 + x86_64 combined) |
| 69 | + |
| 70 | +### Linux |
| 71 | + |
| 72 | +#### Required Software |
| 73 | +1. **Build essentials**: |
| 74 | + ```bash |
| 75 | + # Ubuntu/Debian |
| 76 | + sudo apt-get update |
| 77 | + sudo apt-get install build-essential cmake python3-dev |
| 78 | + |
| 79 | + # RHEL/CentOS/Fedora |
| 80 | + sudo yum groupinstall "Development Tools" |
| 81 | + sudo yum install cmake python3-devel |
| 82 | + ``` |
| 83 | + |
| 84 | +2. **PyBind11**: |
| 85 | + ```bash |
| 86 | + pip install pybind11 |
| 87 | + ``` |
| 88 | + |
| 89 | +#### Architecture Support |
| 90 | +- x86_64 (64-bit Intel/AMD) |
| 91 | +- ARM64/aarch64 (64-bit ARM) |
| 92 | + |
| 93 | +#### Distribution Support |
| 94 | +- Ubuntu/Debian (manylinux2014) |
| 95 | +- RHEL/CentOS/Fedora (manylinux2014) |
| 96 | + |
| 97 | +## Building Native Pybind Bindings |
| 98 | + |
| 99 | +The native bindings are located in `mssql_python/pybind/` and are built using CMake and pybind11. |
| 100 | + |
| 101 | +### Windows |
| 102 | + |
| 103 | +1. **Open Developer Command Prompt for VS 2022** |
| 104 | + |
| 105 | +2. **Navigate to the pybind directory**: |
| 106 | + ```cmd |
| 107 | + cd mssql_python\pybind |
| 108 | + ``` |
| 109 | + |
| 110 | +3. **Run the build script**: |
| 111 | + ```cmd |
| 112 | + build.bat [ARCHITECTURE] |
| 113 | + ``` |
| 114 | + |
| 115 | + Where `[ARCHITECTURE]` can be: |
| 116 | + - `x64` (default if not specified) |
| 117 | + - `x86` |
| 118 | + - `arm64` |
| 119 | + |
| 120 | + Examples: |
| 121 | + ```cmd |
| 122 | + build.bat # Builds for x64 |
| 123 | + build.bat x64 # Builds for x64 |
| 124 | + build.bat arm64 # Builds for ARM64 |
| 125 | + ``` |
| 126 | + |
| 127 | +### macOS |
| 128 | + |
| 129 | +1. **Navigate to the pybind directory**: |
| 130 | + ```bash |
| 131 | + cd mssql_python/pybind |
| 132 | + ``` |
| 133 | + |
| 134 | +2. **Run the build script**: |
| 135 | + ```bash |
| 136 | + ./build.sh |
| 137 | + ``` |
| 138 | + |
| 139 | + This automatically builds universal2 binaries (ARM64 + x86_64). |
| 140 | + |
| 141 | +### Linux |
| 142 | + |
| 143 | +1. **Navigate to the pybind directory**: |
| 144 | + ```bash |
| 145 | + cd mssql_python/pybind |
| 146 | + ``` |
| 147 | + |
| 148 | +2. **Run the build script**: |
| 149 | + ```bash |
| 150 | + ./build.sh |
| 151 | + ``` |
| 152 | + |
| 153 | + The script automatically detects your system architecture. |
| 154 | + |
| 155 | +### Build Output |
| 156 | + |
| 157 | +After successful compilation, you'll find the native binding file in the `mssql_python/` directory: |
| 158 | + |
| 159 | +- **Windows**: `ddbc_bindings.cp{python_version}-{architecture}.pyd` |
| 160 | + - Example: `ddbc_bindings.cp312-amd64.pyd` |
| 161 | +- **macOS**: `ddbc_bindings.cp{python_version}-universal2.so` |
| 162 | + - Example: `ddbc_bindings.cp312-universal2.so` |
| 163 | +- **Linux**: `ddbc_bindings.cp{python_version}-{architecture}.so` |
| 164 | + - Example: `ddbc_bindings.cp312-x86_64.so` |
| 165 | + |
| 166 | +## Creating Python Wheels |
| 167 | + |
| 168 | +After building the native bindings, you can create a Python wheel for distribution. |
| 169 | + |
| 170 | +### Build Wheel |
| 171 | + |
| 172 | +```bash |
| 173 | +python setup.py bdist_wheel |
| 174 | +``` |
| 175 | + |
| 176 | +The wheel will be created in the `dist/` directory with platform-specific tags: |
| 177 | +- **Windows**: `mssql_python-{version}-py3-none-win_{architecture}.whl` |
| 178 | +- **macOS**: `mssql_python-{version}-py3-none-macosx_15_0_universal2.whl` |
| 179 | +- **Linux**: `mssql_python-{version}-py3-none-manylinux2014_{architecture}.whl` |
| 180 | + |
| 181 | +### Install from Wheel |
| 182 | + |
| 183 | +```bash |
| 184 | +pip install dist/mssql_python-{version}-py3-none-{platform}.whl |
| 185 | +``` |
| 186 | + |
| 187 | +### Alternative: Development Installation |
| 188 | + |
| 189 | +For development purposes, you can install the package in editable mode: |
| 190 | + |
| 191 | +```bash |
| 192 | +pip install -e . |
| 193 | +``` |
| 194 | + |
| 195 | +This allows you to make changes to the Python code without reinstalling. |
| 196 | + |
| 197 | +## Running Tests |
| 198 | + |
| 199 | +The test suite uses pytest and requires a Microsoft SQL Server database connection. |
| 200 | + |
| 201 | +### Environment Setup |
| 202 | + |
| 203 | +Set the database connection string environment variable: |
| 204 | + |
| 205 | +#### Windows |
| 206 | +```cmd |
| 207 | +set DB_CONNECTION_STRING="SERVER=your_server;DATABASE=your_database;UID=your_username;PWD=your_password;Encrypt=yes;TrustServerCertificate=yes;" |
| 208 | +``` |
| 209 | + |
| 210 | +#### macOS/Linux |
| 211 | +```bash |
| 212 | +export DB_CONNECTION_STRING="SERVER=your_server;DATABASE=your_database;UID=your_username;PWD=your_password;Encrypt=yes;TrustServerCertificate=yes;" |
| 213 | +``` |
| 214 | + |
| 215 | +### Connection String Examples |
| 216 | + |
| 217 | +#### SQL Server Authentication |
| 218 | +``` |
| 219 | +SERVER=localhost;DATABASE=master;UID=sa;PWD=YourPassword123;Encrypt=yes;TrustServerCertificate=yes; |
| 220 | +``` |
| 221 | + |
| 222 | +#### Windows Authentication (Windows only) |
| 223 | +``` |
| 224 | +SERVER=localhost;DATABASE=master;Trusted_Connection=yes;Encrypt=yes;TrustServerCertificate=yes; |
| 225 | +``` |
| 226 | + |
| 227 | +#### Azure SQL Database |
| 228 | +``` |
| 229 | +SERVER=your-server.database.windows.net;DATABASE=your-database;UID=your-username;PWD=your-password;Encrypt=yes; |
| 230 | +``` |
| 231 | + |
| 232 | +### Running Tests |
| 233 | + |
| 234 | +After setting the environment variable, run the tests: |
| 235 | + |
| 236 | +```bash |
| 237 | +# Run all tests |
| 238 | +python -m pytest |
| 239 | + |
| 240 | +# Run specific test file |
| 241 | +python -m pytest tests/test_003_connection.py |
| 242 | + |
| 243 | +# Run tests with coverage |
| 244 | +python -m pytest --cov=mssql_python |
| 245 | + |
| 246 | +# Run tests with verbose output |
| 247 | +python -m pytest -v |
| 248 | +``` |
| 249 | + |
| 250 | +### Test Structure |
| 251 | + |
| 252 | +The test suite includes: |
| 253 | +- `test_000_dependencies.py` - Dependency validation |
| 254 | +- `test_001_globals.py` - Global functionality tests |
| 255 | +- `test_002_types.py` - Data type handling tests |
| 256 | +- `test_003_connection.py` - Connection management tests |
| 257 | +- `test_004_cursor.py` - Cursor functionality tests |
| 258 | +- `test_005_connection_cursor_lifecycle.py` - Lifecycle tests |
| 259 | +- `test_006_exceptions.py` - Error handling tests |
| 260 | +- `test_007_logging.py` - Logging functionality tests |
| 261 | +- `test_008_auth.py` - Authentication tests |
| 262 | + |
| 263 | +## Troubleshooting |
| 264 | + |
| 265 | +### Common Build Issues |
| 266 | + |
| 267 | +#### Windows |
| 268 | +- **Error**: "Visual Studio not found" |
| 269 | + - **Solution**: Ensure Visual Studio Build Tools 2022 is installed with C++ workload |
| 270 | + - **Alternative**: Use Developer Command Prompt for VS 2022 |
| 271 | + |
| 272 | +- **Error**: "CMake not found" |
| 273 | + - **Solution**: CMake is included with Visual Studio Build Tools, or install separately |
| 274 | + |
| 275 | +#### macOS |
| 276 | +- **Error**: "No such file or directory: 'sql.h'" |
| 277 | + - **Solution**: Install Microsoft ODBC Driver 18 using the brew command above |
| 278 | + |
| 279 | +- **Error**: "xcrun: error: active developer path does not exist" |
| 280 | + - **Solution**: Install Xcode Command Line Tools: `xcode-select --install` |
| 281 | + |
| 282 | +#### Linux |
| 283 | +- **Error**: "Python.h: No such file or directory" |
| 284 | + - **Solution**: Install Python development headers: `sudo apt-get install python3-dev` |
| 285 | + |
| 286 | +- **Error**: "cmake: command not found" |
| 287 | + - **Solution**: Install CMake: `sudo apt-get install cmake` |
| 288 | + |
| 289 | +### Test Issues |
| 290 | + |
| 291 | +- **Error**: "Connection string should not be None" |
| 292 | + - **Solution**: Ensure `DB_CONNECTION_STRING` environment variable is set correctly |
| 293 | + |
| 294 | +- **Error**: "Database connection failed" |
| 295 | + - **Solution**: Verify SQL Server is running and connection string is valid |
| 296 | + - **Check**: Network connectivity and firewall settings |
| 297 | + - **Verify**: Username/password or authentication method |
| 298 | + |
| 299 | +### Architecture Issues |
| 300 | + |
| 301 | +- **Windows**: Ensure you're building for the correct architecture matching your Python installation |
| 302 | +- **macOS**: Universal2 binaries should work on both Intel and Apple Silicon Macs |
| 303 | +- **Linux**: The build script auto-detects architecture, but you can verify with `uname -m` |
| 304 | + |
| 305 | +## Development Workflow |
| 306 | + |
| 307 | +### Recommended Development Process |
| 308 | + |
| 309 | +1. **Set up development environment**: |
| 310 | + ```bash |
| 311 | + git clone https://github.com/microsoft/mssql-python.git |
| 312 | + cd mssql-python |
| 313 | + pip install -r requirements.txt |
| 314 | + ``` |
| 315 | + |
| 316 | +2. **Build native bindings**: |
| 317 | + ```bash |
| 318 | + cd mssql_python/pybind |
| 319 | + # Windows: build.bat |
| 320 | + # macOS/Linux: ./build.sh |
| 321 | + ``` |
| 322 | + |
| 323 | +3. **Install in development mode**: |
| 324 | + ```bash |
| 325 | + cd ../.. |
| 326 | + pip install -e . |
| 327 | + ``` |
| 328 | + |
| 329 | +4. **Set up test database connection**: |
| 330 | + ```bash |
| 331 | + # Windows: set DB_CONNECTION_STRING="..." |
| 332 | + # macOS/Linux: export DB_CONNECTION_STRING="..." |
| 333 | + ``` |
| 334 | + |
| 335 | +5. **Run tests**: |
| 336 | + ```bash |
| 337 | + python -m pytest |
| 338 | + ``` |
| 339 | + |
| 340 | +6. **Make changes and test iteratively**: |
| 341 | + - For Python changes: No rebuild needed (using `-e` install) |
| 342 | + - For C++ changes: Rebuild native bindings and reinstall |
| 343 | + |
| 344 | +### Code Quality |
| 345 | + |
| 346 | +The project uses standard Python development practices: |
| 347 | +- Follow PEP 8 style guidelines |
| 348 | +- Write comprehensive tests for new features |
| 349 | +- Ensure all tests pass before submitting changes |
| 350 | +- Use type hints where appropriate |
| 351 | + |
| 352 | +## Contributing |
| 353 | + |
| 354 | +This project welcomes contributions! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for detailed guidelines on how to contribute to this project. |
| 355 | + |
| 356 | +For questions or support, please create an issue on the [GitHub repository](https://github.com/microsoft/mssql-python/issues). |
0 commit comments