From 7c4693e9645d3e9be4e7dc46c2c166cd9e4d5bf3 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 28 Jul 2025 05:32:01 +0000 Subject: [PATCH 01/11] Initial plan From f78f7663a6203020fae58b5434acef873756f603 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 28 Jul 2025 05:38:23 +0000 Subject: [PATCH 02/11] Add comprehensive BUILDGUIDE.md with cross-platform build and development instructions Co-authored-by: bewithgaurav <8655500+bewithgaurav@users.noreply.github.com> --- BUILDGUIDE.md | 356 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 356 insertions(+) create mode 100644 BUILDGUIDE.md diff --git a/BUILDGUIDE.md b/BUILDGUIDE.md new file mode 100644 index 00000000..202e03f1 --- /dev/null +++ b/BUILDGUIDE.md @@ -0,0 +1,356 @@ +# Build Guide + +This guide provides comprehensive instructions for building and developing the Microsoft Python Driver for SQL Server (`mssql-python`) from source. It covers all supported platforms and explains how to build the native pybind bindings, create Python wheels, and run tests. + +## Prerequisites + +### Python Requirements +- **Python 3.10 or higher** is required +- Use the standard `python` and `pip` commands (no Anaconda or pyenv dependencies) + +### General Dependencies +Install the required Python packages: +```bash +pip install -r requirements.txt +``` + +This will install: +- `pytest` - Testing framework +- `pytest-cov` - Test coverage reporting +- `pybind11` - Python/C++ binding library +- `coverage` - Code coverage measurement +- `unittest-xml-reporting` - XML test reporting +- `setuptools` - Package building tools + +## Platform-Specific Prerequisites + +### Windows + +#### Required Software +1. **Visual Studio Build Tools 2022** or **Visual Studio 2022** with C++ support + - Download from: https://visualstudio.microsoft.com/downloads/ + - During installation, select the **"Desktop development with C++"** workload + - This automatically includes CMake + +2. **PyBind11**: + ```cmd + pip install pybind11 + ``` + +#### Architecture Support +- x64 (64-bit Intel/AMD) +- x86 (32-bit Intel/AMD) +- ARM64 (64-bit ARM) + +### macOS + +#### Required Software +1. **Xcode Command Line Tools**: + ```bash + xcode-select --install + ``` + +2. **CMake and PyBind11**: + ```bash + brew install cmake + pip install pybind11 + ``` + +3. **Microsoft ODBC Driver 18**: + ```bash + brew tap microsoft/mssql-release https://github.com/Microsoft/homebrew-mssql-release + ACCEPT_EULA=Y brew install msodbcsql18 + ``` + + > **Note**: This provides development headers (`sql.h`, `sqlext.h`) and the dynamic library (`libmsodbcsql.18.dylib`) required for building native bindings. + +#### Architecture Support +- Universal2 binaries (ARM64 + x86_64 combined) + +### Linux + +#### Required Software +1. **Build essentials**: + ```bash + # Ubuntu/Debian + sudo apt-get update + sudo apt-get install build-essential cmake python3-dev + + # RHEL/CentOS/Fedora + sudo yum groupinstall "Development Tools" + sudo yum install cmake python3-devel + ``` + +2. **PyBind11**: + ```bash + pip install pybind11 + ``` + +#### Architecture Support +- x86_64 (64-bit Intel/AMD) +- ARM64/aarch64 (64-bit ARM) + +#### Distribution Support +- Ubuntu/Debian (manylinux2014) +- RHEL/CentOS/Fedora (manylinux2014) + +## Building Native Pybind Bindings + +The native bindings are located in `mssql_python/pybind/` and are built using CMake and pybind11. + +### Windows + +1. **Open Developer Command Prompt for VS 2022** + +2. **Navigate to the pybind directory**: + ```cmd + cd mssql_python\pybind + ``` + +3. **Run the build script**: + ```cmd + build.bat [ARCHITECTURE] + ``` + + Where `[ARCHITECTURE]` can be: + - `x64` (default if not specified) + - `x86` + - `arm64` + + Examples: + ```cmd + build.bat # Builds for x64 + build.bat x64 # Builds for x64 + build.bat arm64 # Builds for ARM64 + ``` + +### macOS + +1. **Navigate to the pybind directory**: + ```bash + cd mssql_python/pybind + ``` + +2. **Run the build script**: + ```bash + ./build.sh + ``` + + This automatically builds universal2 binaries (ARM64 + x86_64). + +### Linux + +1. **Navigate to the pybind directory**: + ```bash + cd mssql_python/pybind + ``` + +2. **Run the build script**: + ```bash + ./build.sh + ``` + + The script automatically detects your system architecture. + +### Build Output + +After successful compilation, you'll find the native binding file in the `mssql_python/` directory: + +- **Windows**: `ddbc_bindings.cp{python_version}-{architecture}.pyd` + - Example: `ddbc_bindings.cp312-amd64.pyd` +- **macOS**: `ddbc_bindings.cp{python_version}-universal2.so` + - Example: `ddbc_bindings.cp312-universal2.so` +- **Linux**: `ddbc_bindings.cp{python_version}-{architecture}.so` + - Example: `ddbc_bindings.cp312-x86_64.so` + +## Creating Python Wheels + +After building the native bindings, you can create a Python wheel for distribution. + +### Build Wheel + +```bash +python setup.py bdist_wheel +``` + +The wheel will be created in the `dist/` directory with platform-specific tags: +- **Windows**: `mssql_python-{version}-py3-none-win_{architecture}.whl` +- **macOS**: `mssql_python-{version}-py3-none-macosx_15_0_universal2.whl` +- **Linux**: `mssql_python-{version}-py3-none-manylinux2014_{architecture}.whl` + +### Install from Wheel + +```bash +pip install dist/mssql_python-{version}-py3-none-{platform}.whl +``` + +### Alternative: Development Installation + +For development purposes, you can install the package in editable mode: + +```bash +pip install -e . +``` + +This allows you to make changes to the Python code without reinstalling. + +## Running Tests + +The test suite uses pytest and requires a Microsoft SQL Server database connection. + +### Environment Setup + +Set the database connection string environment variable: + +#### Windows +```cmd +set DB_CONNECTION_STRING="SERVER=your_server;DATABASE=your_database;UID=your_username;PWD=your_password;Encrypt=yes;TrustServerCertificate=yes;" +``` + +#### macOS/Linux +```bash +export DB_CONNECTION_STRING="SERVER=your_server;DATABASE=your_database;UID=your_username;PWD=your_password;Encrypt=yes;TrustServerCertificate=yes;" +``` + +### Connection String Examples + +#### SQL Server Authentication +``` +SERVER=localhost;DATABASE=master;UID=sa;PWD=YourPassword123;Encrypt=yes;TrustServerCertificate=yes; +``` + +#### Windows Authentication (Windows only) +``` +SERVER=localhost;DATABASE=master;Trusted_Connection=yes;Encrypt=yes;TrustServerCertificate=yes; +``` + +#### Azure SQL Database +``` +SERVER=your-server.database.windows.net;DATABASE=your-database;UID=your-username;PWD=your-password;Encrypt=yes; +``` + +### Running Tests + +After setting the environment variable, run the tests: + +```bash +# Run all tests +python -m pytest + +# Run specific test file +python -m pytest tests/test_003_connection.py + +# Run tests with coverage +python -m pytest --cov=mssql_python + +# Run tests with verbose output +python -m pytest -v +``` + +### Test Structure + +The test suite includes: +- `test_000_dependencies.py` - Dependency validation +- `test_001_globals.py` - Global functionality tests +- `test_002_types.py` - Data type handling tests +- `test_003_connection.py` - Connection management tests +- `test_004_cursor.py` - Cursor functionality tests +- `test_005_connection_cursor_lifecycle.py` - Lifecycle tests +- `test_006_exceptions.py` - Error handling tests +- `test_007_logging.py` - Logging functionality tests +- `test_008_auth.py` - Authentication tests + +## Troubleshooting + +### Common Build Issues + +#### Windows +- **Error**: "Visual Studio not found" + - **Solution**: Ensure Visual Studio Build Tools 2022 is installed with C++ workload + - **Alternative**: Use Developer Command Prompt for VS 2022 + +- **Error**: "CMake not found" + - **Solution**: CMake is included with Visual Studio Build Tools, or install separately + +#### macOS +- **Error**: "No such file or directory: 'sql.h'" + - **Solution**: Install Microsoft ODBC Driver 18 using the brew command above + +- **Error**: "xcrun: error: active developer path does not exist" + - **Solution**: Install Xcode Command Line Tools: `xcode-select --install` + +#### Linux +- **Error**: "Python.h: No such file or directory" + - **Solution**: Install Python development headers: `sudo apt-get install python3-dev` + +- **Error**: "cmake: command not found" + - **Solution**: Install CMake: `sudo apt-get install cmake` + +### Test Issues + +- **Error**: "Connection string should not be None" + - **Solution**: Ensure `DB_CONNECTION_STRING` environment variable is set correctly + +- **Error**: "Database connection failed" + - **Solution**: Verify SQL Server is running and connection string is valid + - **Check**: Network connectivity and firewall settings + - **Verify**: Username/password or authentication method + +### Architecture Issues + +- **Windows**: Ensure you're building for the correct architecture matching your Python installation +- **macOS**: Universal2 binaries should work on both Intel and Apple Silicon Macs +- **Linux**: The build script auto-detects architecture, but you can verify with `uname -m` + +## Development Workflow + +### Recommended Development Process + +1. **Set up development environment**: + ```bash + git clone https://github.com/microsoft/mssql-python.git + cd mssql-python + pip install -r requirements.txt + ``` + +2. **Build native bindings**: + ```bash + cd mssql_python/pybind + # Windows: build.bat + # macOS/Linux: ./build.sh + ``` + +3. **Install in development mode**: + ```bash + cd ../.. + pip install -e . + ``` + +4. **Set up test database connection**: + ```bash + # Windows: set DB_CONNECTION_STRING="..." + # macOS/Linux: export DB_CONNECTION_STRING="..." + ``` + +5. **Run tests**: + ```bash + python -m pytest + ``` + +6. **Make changes and test iteratively**: + - For Python changes: No rebuild needed (using `-e` install) + - For C++ changes: Rebuild native bindings and reinstall + +### Code Quality + +The project uses standard Python development practices: +- Follow PEP 8 style guidelines +- Write comprehensive tests for new features +- Ensure all tests pass before submitting changes +- Use type hints where appropriate + +## Contributing + +This project welcomes contributions! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for detailed guidelines on how to contribute to this project. + +For questions or support, please create an issue on the [GitHub repository](https://github.com/microsoft/mssql-python/issues). \ No newline at end of file From be8e17fa69526adf8d666fe0600826d061eb5387 Mon Sep 17 00:00:00 2001 From: Gaurav Sharma Date: Mon, 28 Jul 2025 11:22:16 +0530 Subject: [PATCH 03/11] Update BUILDGUIDE.md --- BUILDGUIDE.md | 402 +++++++++++++++----------------------------------- 1 file changed, 121 insertions(+), 281 deletions(-) diff --git a/BUILDGUIDE.md b/BUILDGUIDE.md index 202e03f1..c7031fe5 100644 --- a/BUILDGUIDE.md +++ b/BUILDGUIDE.md @@ -1,356 +1,196 @@ -# Build Guide +# Build Guide for Contributors -This guide provides comprehensive instructions for building and developing the Microsoft Python Driver for SQL Server (`mssql-python`) from source. It covers all supported platforms and explains how to build the native pybind bindings, create Python wheels, and run tests. +Welcome to the development build guide for the **mssql-python** project! +This guide will help you set up your environment, build the native bindings, and package the project as a Python wheel. -## Prerequisites - -### Python Requirements -- **Python 3.10 or higher** is required -- Use the standard `python` and `pip` commands (no Anaconda or pyenv dependencies) +--- -### General Dependencies -Install the required Python packages: -```bash -pip install -r requirements.txt -``` +## Table of Contents -This will install: -- `pytest` - Testing framework -- `pytest-cov` - Test coverage reporting -- `pybind11` - Python/C++ binding library -- `coverage` - Code coverage measurement -- `unittest-xml-reporting` - XML test reporting -- `setuptools` - Package building tools +- [Prerequisites](#prerequisites) +- [Platform-Specific Setup](#platform-specific-setup) + - [Windows](#windows) + - [macOS](#macos) + - [Linux](#linux) +- [Building Native Bindings](#building-native-bindings) +- [Building the Python Wheel (.whl)](#building-the-python-wheel-whl) +- [Running Tests](#running-tests) +- [Directory Structure](#directory-structure) +- [Troubleshooting](#troubleshooting) -## Platform-Specific Prerequisites +--- -### Windows +## Prerequisites -#### Required Software -1. **Visual Studio Build Tools 2022** or **Visual Studio 2022** with C++ support - - Download from: https://visualstudio.microsoft.com/downloads/ - - During installation, select the **"Desktop development with C++"** workload - - This automatically includes CMake +- **Python:** Minimum supported version is 3.10. + Ensure `python` and `pip` commands refer to your Python 3.10+ installation. +- **pybind11:** Used for C++/Python bindings. +- **CMake:** For Unix and macOS builds. +- **Microsoft ODBC Driver:** For packaging driver dependencies and header files such as `sql.h`, `sqlext.h` etc. +- **setuptools, wheel, pytest:** For building and testing (`pip install setuptools wheel pytest`). -2. **PyBind11**: - ```cmd - pip install pybind11 - ``` +--- -#### Architecture Support -- x64 (64-bit Intel/AMD) -- x86 (32-bit Intel/AMD) -- ARM64 (64-bit ARM) +## Platform-Specific Setup -### macOS +### Windows -#### Required Software -1. **Xcode Command Line Tools**: +1. **Install Python** (3.10+ from [python.org](https://www.python.org/downloads/)). +2. **Install Visual Studio Build Tools** + - Include the “Desktop development with C++” workload. + - CMake is included by default. +3. **Install Microsoft ODBC Driver for SQL Server:** + [Download here](https://docs.microsoft.com/en-us/sql/connect/odbc/download-odbc-driver-for-sql-server). +4. **Install required Python packages:** ```bash - xcode-select --install + # Will install pybind11, setuptools etc. + pip install -r requirements.txt ``` -2. **CMake and PyBind11**: +### macOS + +1. **Install Python** (3.10+ from [python.org](https://www.python.org/downloads/)). +2. **Install CMake:** ```bash brew install cmake - pip install pybind11 ``` - -3. **Microsoft ODBC Driver 18**: +3. **Install Microsoft ODBC Driver for SQL Server:** ```bash + /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)" brew tap microsoft/mssql-release https://github.com/Microsoft/homebrew-mssql-release - ACCEPT_EULA=Y brew install msodbcsql18 + brew update + HOMEBREW_ACCEPT_EULA=Y brew install msodbcsql18 mssql-tools18 + ``` +4. **Install Python requirements:** + ```bash + # Will install pybind11, setuptools etc. + pip install -r requirements.txt ``` - - > **Note**: This provides development headers (`sql.h`, `sqlext.h`) and the dynamic library (`libmsodbcsql.18.dylib`) required for building native bindings. - -#### Architecture Support -- Universal2 binaries (ARM64 + x86_64 combined) ### Linux -#### Required Software -1. **Build essentials**: +1. **Install Python and development tools:** ```bash - # Ubuntu/Debian sudo apt-get update - sudo apt-get install build-essential cmake python3-dev - - # RHEL/CentOS/Fedora - sudo yum groupinstall "Development Tools" - sudo yum install cmake python3-devel + sudo apt-get install python3.10 python3.10-dev python3-pip build-essential cmake ``` - -2. **PyBind11**: + Ensure `python` and `pip` refer to Python 3.10. +2. **Install Microsoft ODBC Driver for SQL Server:** + Follow [official instructions](https://docs.microsoft.com/en-us/sql/connect/odbc/linux-mac/installing-the-microsoft-odbc-driver-for-sql-server). +3. **Install Python packages:** ```bash - pip install pybind11 + # Will install pybind11, setuptools etc. + pip install -r requirements.txt ``` -#### Architecture Support -- x86_64 (64-bit Intel/AMD) -- ARM64/aarch64 (64-bit ARM) +--- -#### Distribution Support -- Ubuntu/Debian (manylinux2014) -- RHEL/CentOS/Fedora (manylinux2014) +## Building Native Bindings -## Building Native Pybind Bindings - -The native bindings are located in `mssql_python/pybind/` and are built using CMake and pybind11. +The native bindings are in the `pybind` directory. ### Windows -1. **Open Developer Command Prompt for VS 2022** - -2. **Navigate to the pybind directory**: - ```cmd - cd mssql_python\pybind - ``` - -3. **Run the build script**: - ```cmd - build.bat [ARCHITECTURE] - ``` - - Where `[ARCHITECTURE]` can be: - - `x64` (default if not specified) - - `x86` - - `arm64` - - Examples: - ```cmd - build.bat # Builds for x64 - build.bat x64 # Builds for x64 - build.bat arm64 # Builds for ARM64 - ``` +Open a **Developer Command Prompt for VS** and run: -### macOS - -1. **Navigate to the pybind directory**: - ```bash - cd mssql_python/pybind - ``` - -2. **Run the build script**: - ```bash - ./build.sh - ``` - - This automatically builds universal2 binaries (ARM64 + x86_64). - -### Linux +```bash +cd pybind +build.bat +``` -1. **Navigate to the pybind directory**: - ```bash - cd mssql_python/pybind - ``` +This will: +- Clean previous builds +- Configure with CMake +- Build the extension +- Copy the generated `.pyd` file to the correct location -2. **Run the build script**: - ```bash - ./build.sh - ``` +### macOS & Linux - The script automatically detects your system architecture. +```bash +cd pybind +./build.sh +``` -### Build Output +This will: +- Clean previous builds +- Configure with CMake +- Build the extension +- Copy the generated `.so` file to the correct location -After successful compilation, you'll find the native binding file in the `mssql_python/` directory: +--- -- **Windows**: `ddbc_bindings.cp{python_version}-{architecture}.pyd` - - Example: `ddbc_bindings.cp312-amd64.pyd` -- **macOS**: `ddbc_bindings.cp{python_version}-universal2.so` - - Example: `ddbc_bindings.cp312-universal2.so` -- **Linux**: `ddbc_bindings.cp{python_version}-{architecture}.so` - - Example: `ddbc_bindings.cp312-x86_64.so` +## Building the Python Wheel (.whl) -## Creating Python Wheels +The wheel includes the native bindings. +**You must build the native bindings first** (see above). -After building the native bindings, you can create a Python wheel for distribution. +### Windows -### Build Wheel +From the project root: ```bash python setup.py bdist_wheel ``` -The wheel will be created in the `dist/` directory with platform-specific tags: -- **Windows**: `mssql_python-{version}-py3-none-win_{architecture}.whl` -- **macOS**: `mssql_python-{version}-py3-none-macosx_15_0_universal2.whl` -- **Linux**: `mssql_python-{version}-py3-none-manylinux2014_{architecture}.whl` +The wheel file will be created in the `dist/` directory. -### Install from Wheel +### macOS & Linux -```bash -pip install dist/mssql_python-{version}-py3-none-{platform}.whl -``` - -### Alternative: Development Installation - -For development purposes, you can install the package in editable mode: +From the project root: ```bash -pip install -e . -``` - -This allows you to make changes to the Python code without reinstalling. +# Build the bindings first! +cd pybind +./build.sh +cd .. -## Running Tests - -The test suite uses pytest and requires a Microsoft SQL Server database connection. - -### Environment Setup - -Set the database connection string environment variable: - -#### Windows -```cmd -set DB_CONNECTION_STRING="SERVER=your_server;DATABASE=your_database;UID=your_username;PWD=your_password;Encrypt=yes;TrustServerCertificate=yes;" +# Then build the wheel: +python setup.py bdist_wheel ``` -#### macOS/Linux -```bash -export DB_CONNECTION_STRING="SERVER=your_server;DATABASE=your_database;UID=your_username;PWD=your_password;Encrypt=yes;TrustServerCertificate=yes;" -``` +The wheel file will be created in the `dist/` directory. -### Connection String Examples +--- -#### SQL Server Authentication -``` -SERVER=localhost;DATABASE=master;UID=sa;PWD=YourPassword123;Encrypt=yes;TrustServerCertificate=yes; -``` +## Running Tests -#### Windows Authentication (Windows only) -``` -SERVER=localhost;DATABASE=master;Trusted_Connection=yes;Encrypt=yes;TrustServerCertificate=yes; -``` +Tests require a database connection string. +Set the `DB_CONNECTION_STRING` environment variable before running tests: -#### Azure SQL Database -``` -SERVER=your-server.database.windows.net;DATABASE=your-database;UID=your-username;PWD=your-password;Encrypt=yes; +### Windows (Command Prompt) +```cmd +set DB_CONNECTION_STRING=your-connection-string-here +python -m pytest -v ``` -### Running Tests - -After setting the environment variable, run the tests: - +### macOS & Linux (bash/zsh) ```bash -# Run all tests -python -m pytest - -# Run specific test file -python -m pytest tests/test_003_connection.py - -# Run tests with coverage -python -m pytest --cov=mssql_python - -# Run tests with verbose output +export DB_CONNECTION_STRING=your-connection-string-here python -m pytest -v ``` -### Test Structure - -The test suite includes: -- `test_000_dependencies.py` - Dependency validation -- `test_001_globals.py` - Global functionality tests -- `test_002_types.py` - Data type handling tests -- `test_003_connection.py` - Connection management tests -- `test_004_cursor.py` - Cursor functionality tests -- `test_005_connection_cursor_lifecycle.py` - Lifecycle tests -- `test_006_exceptions.py` - Error handling tests -- `test_007_logging.py` - Logging functionality tests -- `test_008_auth.py` - Authentication tests - -## Troubleshooting - -### Common Build Issues - -#### Windows -- **Error**: "Visual Studio not found" - - **Solution**: Ensure Visual Studio Build Tools 2022 is installed with C++ workload - - **Alternative**: Use Developer Command Prompt for VS 2022 +--- -- **Error**: "CMake not found" - - **Solution**: CMake is included with Visual Studio Build Tools, or install separately +## Directory Structure -#### macOS -- **Error**: "No such file or directory: 'sql.h'" - - **Solution**: Install Microsoft ODBC Driver 18 using the brew command above +- `pybind/` — Native C++/pybind11 bindings and platform build scripts +- `mssql_python/` — Python package source +- `tests/` — Test suite +- `dist/` — Built wheel packages -- **Error**: "xcrun: error: active developer path does not exist" - - **Solution**: Install Xcode Command Line Tools: `xcode-select --install` +--- -#### Linux -- **Error**: "Python.h: No such file or directory" - - **Solution**: Install Python development headers: `sudo apt-get install python3-dev` - -- **Error**: "cmake: command not found" - - **Solution**: Install CMake: `sudo apt-get install cmake` - -### Test Issues - -- **Error**: "Connection string should not be None" - - **Solution**: Ensure `DB_CONNECTION_STRING` environment variable is set correctly - -- **Error**: "Database connection failed" - - **Solution**: Verify SQL Server is running and connection string is valid - - **Check**: Network connectivity and firewall settings - - **Verify**: Username/password or authentication method - -### Architecture Issues - -- **Windows**: Ensure you're building for the correct architecture matching your Python installation -- **macOS**: Universal2 binaries should work on both Intel and Apple Silicon Macs -- **Linux**: The build script auto-detects architecture, but you can verify with `uname -m` - -## Development Workflow - -### Recommended Development Process - -1. **Set up development environment**: - ```bash - git clone https://github.com/microsoft/mssql-python.git - cd mssql-python - pip install -r requirements.txt - ``` - -2. **Build native bindings**: - ```bash - cd mssql_python/pybind - # Windows: build.bat - # macOS/Linux: ./build.sh - ``` - -3. **Install in development mode**: - ```bash - cd ../.. - pip install -e . - ``` - -4. **Set up test database connection**: - ```bash - # Windows: set DB_CONNECTION_STRING="..." - # macOS/Linux: export DB_CONNECTION_STRING="..." - ``` - -5. **Run tests**: - ```bash - python -m pytest - ``` - -6. **Make changes and test iteratively**: - - For Python changes: No rebuild needed (using `-e` install) - - For C++ changes: Rebuild native bindings and reinstall +## Troubleshooting -### Code Quality +- Ensure all prerequisites are installed and on your PATH. +- If a build fails, clean up old artifacts and try again (`pybind/build.bat clean` or `./build.sh clean`). +- For wheel issues, ensure the native binding (`.pyd` or `.so`) is present in the expected location before building the wheel. +- For test failures, double-check your `DB_CONNECTION_STRING`. -The project uses standard Python development practices: -- Follow PEP 8 style guidelines -- Write comprehensive tests for new features -- Ensure all tests pass before submitting changes -- Use type hints where appropriate +--- -## Contributing +For more details on the native bindings, see [`pybind/README.md`](pybind/README.md). -This project welcomes contributions! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for detailed guidelines on how to contribute to this project. +--- -For questions or support, please create an issue on the [GitHub repository](https://github.com/microsoft/mssql-python/issues). \ No newline at end of file +Happy coding! From d1517995cc500a76ecda0d67ceaadd0b1e5019d9 Mon Sep 17 00:00:00 2001 From: Gaurav Sharma Date: Mon, 28 Jul 2025 11:30:51 +0530 Subject: [PATCH 04/11] Update BUILDGUIDE.md --- BUILDGUIDE.md | 15 +++++---------- 1 file changed, 5 insertions(+), 10 deletions(-) diff --git a/BUILDGUIDE.md b/BUILDGUIDE.md index c7031fe5..53df70ee 100644 --- a/BUILDGUIDE.md +++ b/BUILDGUIDE.md @@ -55,13 +55,8 @@ This guide will help you set up your environment, build the native bindings, and brew install cmake ``` 3. **Install Microsoft ODBC Driver for SQL Server:** - ```bash - /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)" - brew tap microsoft/mssql-release https://github.com/Microsoft/homebrew-mssql-release - brew update - HOMEBREW_ACCEPT_EULA=Y brew install msodbcsql18 mssql-tools18 - ``` -4. **Install Python requirements:** + - Follow [official instructions](https://learn.microsoft.com/en-us/sql/connect/odbc/linux-mac/install-microsoft-odbc-driver-sql-server-macos). +5. **Install Python requirements:** ```bash # Will install pybind11, setuptools etc. pip install -r requirements.txt @@ -72,11 +67,11 @@ This guide will help you set up your environment, build the native bindings, and 1. **Install Python and development tools:** ```bash sudo apt-get update - sudo apt-get install python3.10 python3.10-dev python3-pip build-essential cmake + sudo apt-get install python3 python3-dev python3-pip build-essential cmake ``` - Ensure `python` and `pip` refer to Python 3.10. + Ensure `python` and `pip` refer to Python 3.10+. 2. **Install Microsoft ODBC Driver for SQL Server:** - Follow [official instructions](https://docs.microsoft.com/en-us/sql/connect/odbc/linux-mac/installing-the-microsoft-odbc-driver-for-sql-server). + - Follow [official instructions](https://docs.microsoft.com/en-us/sql/connect/odbc/linux-mac/installing-the-microsoft-odbc-driver-for-sql-server). 3. **Install Python packages:** ```bash # Will install pybind11, setuptools etc. From 42bfa8a305437311b18cb14ba7b22f923b0fd5b3 Mon Sep 17 00:00:00 2001 From: Gaurav Sharma Date: Mon, 28 Jul 2025 11:32:17 +0530 Subject: [PATCH 05/11] Update BUILDGUIDE.md --- BUILDGUIDE.md | 38 +++++++++++++++++++------------------- 1 file changed, 19 insertions(+), 19 deletions(-) diff --git a/BUILDGUIDE.md b/BUILDGUIDE.md index 53df70ee..b3c76058 100644 --- a/BUILDGUIDE.md +++ b/BUILDGUIDE.md @@ -114,6 +114,25 @@ This will: --- +## Running Tests + +Tests require a database connection string. +Set the `DB_CONNECTION_STRING` environment variable before running tests: + +### Windows (Command Prompt) +```cmd +set DB_CONNECTION_STRING=your-connection-string-here +python -m pytest -v +``` + +### macOS & Linux (bash/zsh) +```bash +export DB_CONNECTION_STRING=your-connection-string-here +python -m pytest -v +``` + +--- + ## Building the Python Wheel (.whl) The wheel includes the native bindings. @@ -147,25 +166,6 @@ The wheel file will be created in the `dist/` directory. --- -## Running Tests - -Tests require a database connection string. -Set the `DB_CONNECTION_STRING` environment variable before running tests: - -### Windows (Command Prompt) -```cmd -set DB_CONNECTION_STRING=your-connection-string-here -python -m pytest -v -``` - -### macOS & Linux (bash/zsh) -```bash -export DB_CONNECTION_STRING=your-connection-string-here -python -m pytest -v -``` - ---- - ## Directory Structure - `pybind/` — Native C++/pybind11 bindings and platform build scripts From 8b1a67b037399e19f5c200fc28ad55b2440ada23 Mon Sep 17 00:00:00 2001 From: Gaurav Sharma Date: Mon, 28 Jul 2025 12:23:25 +0530 Subject: [PATCH 06/11] Update BUILDGUIDE.md --- BUILDGUIDE.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/BUILDGUIDE.md b/BUILDGUIDE.md index b3c76058..c53fcf94 100644 --- a/BUILDGUIDE.md +++ b/BUILDGUIDE.md @@ -56,7 +56,7 @@ This guide will help you set up your environment, build the native bindings, and ``` 3. **Install Microsoft ODBC Driver for SQL Server:** - Follow [official instructions](https://learn.microsoft.com/en-us/sql/connect/odbc/linux-mac/install-microsoft-odbc-driver-sql-server-macos). -5. **Install Python requirements:** +4. **Install Python requirements:** ```bash # Will install pybind11, setuptools etc. pip install -r requirements.txt From d9a7ef7b0f120c0c656ca33e4bf7cb8b006f9825 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 28 Jul 2025 19:10:19 +0000 Subject: [PATCH 07/11] Fix broken links and correct pybind directory paths in BUILDGUIDE.md Co-authored-by: dlevy-msft-sql <194277063+dlevy-msft-sql@users.noreply.github.com> --- BUILDGUIDE.md | 69 ++++++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 60 insertions(+), 9 deletions(-) diff --git a/BUILDGUIDE.md b/BUILDGUIDE.md index c53fcf94..6b252065 100644 --- a/BUILDGUIDE.md +++ b/BUILDGUIDE.md @@ -7,6 +7,7 @@ This guide will help you set up your environment, build the native bindings, and ## Table of Contents +- [Getting Started](#getting-started) - [Prerequisites](#prerequisites) - [Platform-Specific Setup](#platform-specific-setup) - [Windows](#windows) @@ -15,11 +16,29 @@ This guide will help you set up your environment, build the native bindings, and - [Building Native Bindings](#building-native-bindings) - [Building the Python Wheel (.whl)](#building-the-python-wheel-whl) - [Running Tests](#running-tests) +- [Setting Up a Test Database (Optional)](#setting-up-a-test-database-optional) - [Directory Structure](#directory-structure) - [Troubleshooting](#troubleshooting) --- +## Getting Started + +To contribute to this project, you'll need to fork and clone the repository: + +1. **Fork the repository** on GitHub by clicking the "Fork" button on the [mssql-python repository page](https://github.com/microsoft/mssql-python). +2. **Clone your fork** to your local machine: + ```bash + git clone https://github.com/YOUR-USERNAME/mssql-python.git + cd mssql-python + ``` +3. **Set up the upstream remote** to keep your fork in sync: + ```bash + git remote add upstream https://github.com/microsoft/mssql-python.git + ``` + +--- + ## Prerequisites - **Python:** Minimum supported version is 3.10. @@ -39,6 +58,7 @@ This guide will help you set up your environment, build the native bindings, and 2. **Install Visual Studio Build Tools** - Include the “Desktop development with C++” workload. - CMake is included by default. + - **Alternative for VS Code users:** If you already have VS Code installed, you can configure it for C++ development by following [this guide](https://code.visualstudio.com/docs/cpp/config-msvc). 3. **Install Microsoft ODBC Driver for SQL Server:** [Download here](https://docs.microsoft.com/en-us/sql/connect/odbc/download-odbc-driver-for-sql-server). 4. **Install required Python packages:** @@ -82,14 +102,14 @@ This guide will help you set up your environment, build the native bindings, and ## Building Native Bindings -The native bindings are in the `pybind` directory. +The native bindings are in the `mssql_python/pybind` directory. ### Windows Open a **Developer Command Prompt for VS** and run: ```bash -cd pybind +cd mssql_python/pybind build.bat ``` @@ -102,7 +122,7 @@ This will: ### macOS & Linux ```bash -cd pybind +cd mssql_python/pybind ./build.sh ``` @@ -116,17 +136,23 @@ This will: ## Running Tests -Tests require a database connection string. +Tests require a database connection string and must be run from the project root directory. Set the `DB_CONNECTION_STRING` environment variable before running tests: ### Windows (Command Prompt) ```cmd +# If you're in mssql_python/pybind/, navigate back to the project root: +cd ../.. + set DB_CONNECTION_STRING=your-connection-string-here python -m pytest -v ``` ### macOS & Linux (bash/zsh) ```bash +# If you're in mssql_python/pybind/, navigate back to the project root: +cd ../.. + export DB_CONNECTION_STRING=your-connection-string-here python -m pytest -v ``` @@ -154,9 +180,9 @@ From the project root: ```bash # Build the bindings first! -cd pybind +cd mssql_python/pybind ./build.sh -cd .. +cd ../.. # Then build the wheel: python setup.py bdist_wheel @@ -168,23 +194,48 @@ The wheel file will be created in the `dist/` directory. ## Directory Structure -- `pybind/` — Native C++/pybind11 bindings and platform build scripts +- `mssql_python/pybind/` — Native C++/pybind11 bindings and platform build scripts - `mssql_python/` — Python package source - `tests/` — Test suite - `dist/` — Built wheel packages --- +## Setting Up a Test Database (Optional) + +If you don't have access to a SQL Server instance, you can quickly set up a containerized SQL Server using go-sqlcmd: + +### Windows +```bash +# Install Docker Desktop and sqlcmd +winget install Docker.DockerDesktop +``` +Configure Docker, accept EULA, etc., then open a new terminal window: +```bash +winget install sqlcmd +``` +Open a new window to get new path variables: +```bash +sqlcmd create mssql --name mssql-python --accept-eula tag 2025-latest --using https://github.com/Microsoft/sql-server-samples/releases/download/wide-world-importers-v1.0/WideWorldImporters-Full.bak +sqlcmd config connection-strings +``` +Copy the ODBC connection string and remove the driver clause before storing it in your `DB_CONNECTION_STRING` environment variable. + +### macOS & Linux +Similar commands are available for macOS and Linux. See the [go-sqlcmd documentation](https://learn.microsoft.com/en-us/sql/tools/sqlcmd/quickstart-sqlcmd-create-container?view=sql-server-ver17) for platform-specific instructions. + +--- + ## Troubleshooting - Ensure all prerequisites are installed and on your PATH. -- If a build fails, clean up old artifacts and try again (`pybind/build.bat clean` or `./build.sh clean`). +- If a build fails, clean up old artifacts and try again (`mssql_python/pybind/build.bat clean` or `./build.sh clean`). - For wheel issues, ensure the native binding (`.pyd` or `.so`) is present in the expected location before building the wheel. - For test failures, double-check your `DB_CONNECTION_STRING`. --- -For more details on the native bindings, see [`pybind/README.md`](pybind/README.md). +For more details on the native bindings, see [`mssql_python/pybind/README.md`](mssql_python/pybind/README.md). --- From b95713eb86ae4b7c5bc543e782bf3e49efc868bd Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 28 Jul 2025 19:20:03 +0000 Subject: [PATCH 08/11] Add VS Code-specific fork and clone instructions with GitHub extension setup Co-authored-by: dlevy-msft-sql <194277063+dlevy-msft-sql@users.noreply.github.com> --- BUILDGUIDE.md | 21 ++++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-) diff --git a/BUILDGUIDE.md b/BUILDGUIDE.md index 6b252065..6310a8be 100644 --- a/BUILDGUIDE.md +++ b/BUILDGUIDE.md @@ -24,7 +24,9 @@ This guide will help you set up your environment, build the native bindings, and ## Getting Started -To contribute to this project, you'll need to fork and clone the repository: +To contribute to this project, you'll need to fork and clone the repository. You can do this using either the command line or Visual Studio Code. + +### Option 1: Command Line 1. **Fork the repository** on GitHub by clicking the "Fork" button on the [mssql-python repository page](https://github.com/microsoft/mssql-python). 2. **Clone your fork** to your local machine: @@ -37,6 +39,23 @@ To contribute to this project, you'll need to fork and clone the repository: git remote add upstream https://github.com/microsoft/mssql-python.git ``` +### Option 2: Visual Studio Code + +1. **Install the GitHub extension** in VS Code: + - Open VS Code + - Go to the Extensions view (Ctrl+Shift+X) + - Search for "GitHub Pull Requests and Issues" and install it +2. **Fork and clone the repository**: + - Navigate to the [mssql-python repository page](https://github.com/microsoft/mssql-python) + - Click "Create new fork" to create a fork in your GitHub account + - Open VS Code + - Open the Command Palette (Ctrl+Shift+P) + - Type "Git: Clone" and select it + - Select "Clone from GitHub" + - Search for and select your forked repository + - Choose a local directory to clone to +3. **The upstream remote will be set up automatically** when you fork through GitHub. + --- ## Prerequisites From b90a5aed501f5671821d1a953e30d76dec40a894 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 28 Jul 2025 19:24:16 +0000 Subject: [PATCH 09/11] Add macOS and Linux specific commands for database setup Co-authored-by: dlevy-msft-sql <194277063+dlevy-msft-sql@users.noreply.github.com> --- BUILDGUIDE.md | 35 +++++++++++++++++++++++++++++++++-- 1 file changed, 33 insertions(+), 2 deletions(-) diff --git a/BUILDGUIDE.md b/BUILDGUIDE.md index 6310a8be..bcc4ef1c 100644 --- a/BUILDGUIDE.md +++ b/BUILDGUIDE.md @@ -240,8 +240,39 @@ sqlcmd config connection-strings ``` Copy the ODBC connection string and remove the driver clause before storing it in your `DB_CONNECTION_STRING` environment variable. -### macOS & Linux -Similar commands are available for macOS and Linux. See the [go-sqlcmd documentation](https://learn.microsoft.com/en-us/sql/tools/sqlcmd/quickstart-sqlcmd-create-container?view=sql-server-ver17) for platform-specific instructions. +### macOS +```bash +# Install Docker Desktop and sqlcmd +brew install --cask docker +brew install sqlcmd +``` +Start Docker Desktop, then: +```bash +sqlcmd create mssql --name mssql-python --accept-eula tag 2025-latest --using https://github.com/Microsoft/sql-server-samples/releases/download/wide-world-importers-v1.0/WideWorldImporters-Full.bak +sqlcmd config connection-strings +``` +Copy the ODBC connection string and remove the driver clause before storing it in your `DB_CONNECTION_STRING` environment variable. + +### Linux +```bash +# Install Docker and sqlcmd (Ubuntu/Debian) +sudo apt-get update +sudo apt-get install docker.io +sudo systemctl start docker +sudo systemctl enable docker + +# Install sqlcmd +curl -fsSL https://packages.microsoft.com/keys/microsoft.asc | sudo gpg --dearmor -o /usr/share/keyrings/microsoft-prod.gpg +echo "deb [arch=amd64,arm64,armhf signed-by=/usr/share/keyrings/microsoft-prod.gpg] https://packages.microsoft.com/repos/microsoft-ubuntu-$(lsb_release -rs)-prod $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/mssql-release.list +sudo apt-get update +sudo apt-get install sqlcmd +``` +Then create the SQL Server container: +```bash +sudo sqlcmd create mssql --name mssql-python --accept-eula tag 2025-latest --using https://github.com/Microsoft/sql-server-samples/releases/download/wide-world-importers-v1.0/WideWorldImporters-Full.bak +sqlcmd config connection-strings +``` +Copy the ODBC connection string and remove the driver clause before storing it in your `DB_CONNECTION_STRING` environment variable. --- From 2ef99b6d98a124a88293ddd6466a9da1ad062783 Mon Sep 17 00:00:00 2001 From: David Levy Date: Mon, 4 Aug 2025 10:52:54 -0500 Subject: [PATCH 10/11] Update BUILDGUIDE.md Adding tip on environment variables --- BUILDGUIDE.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/BUILDGUIDE.md b/BUILDGUIDE.md index bcc4ef1c..57b9aede 100644 --- a/BUILDGUIDE.md +++ b/BUILDGUIDE.md @@ -158,6 +158,9 @@ This will: Tests require a database connection string and must be run from the project root directory. Set the `DB_CONNECTION_STRING` environment variable before running tests: +> [!TIP] +> If you are using VS Code you may have to close all open windows for the new environment variable to be visible. + ### Windows (Command Prompt) ```cmd # If you're in mssql_python/pybind/, navigate back to the project root: From ed0ecc8f75ef048e2db336f2d654b87660e4692a Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 4 Aug 2025 15:56:36 +0000 Subject: [PATCH 11/11] Add reboot note for environment variable troubleshooting in BUILDGUIDE.md Co-authored-by: dlevy-msft-sql <194277063+dlevy-msft-sql@users.noreply.github.com> --- BUILDGUIDE.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/BUILDGUIDE.md b/BUILDGUIDE.md index 57b9aede..ec5d3937 100644 --- a/BUILDGUIDE.md +++ b/BUILDGUIDE.md @@ -159,7 +159,7 @@ Tests require a database connection string and must be run from the project root Set the `DB_CONNECTION_STRING` environment variable before running tests: > [!TIP] -> If you are using VS Code you may have to close all open windows for the new environment variable to be visible. +> If you are using VS Code you may have to close all open instances of the IDE for the new environment variable to be visible. If you are still getting errors about invalid connection strings, you may need to reboot your system. ### Windows (Command Prompt) ```cmd