EB_Dashboard/DOCUMENTATION/DOCUMENTATION_98_USER_GUIDE.md

Endobest Dashboard - User Guide

Document Version: 1.0 Last Updated: 2025-11-08 Audience: End Users, Clinical Staff, Data Analysts Language: English

---

Quick Start

The Endobest Dashboard collects patient inclusion data and generates reports. It supports three modes:

Mode 1: Normal Collection (Full Process)

python eb_dashboard.py

• Authenticates with server
• Collects data from all organizations
• Runs quality checks
• Generates JSON files
• Exports to Excel (if configured)
• Takes 5-30 minutes depending on data volume

Mode 2: Excel Export Only (Fast)

python eb_dashboard.py --excel-only

• Skips data collection
• Uses existing JSON data files
• Generates Excel reports
• Takes 5-15 seconds
• Useful for: Reconfiguring reports, testing templates, regenerating with new rules

Mode 3: Quality Checks Only

python eb_dashboard.py --check-only

• Loads existing JSON data
• Runs quality checks
• No export
• Useful for: Verifying data integrity, debugging

---

What Happens When You Run the Script

Normal Mode Flow

1. Login (provides credentials)
   ↓
2. Configuration Loading (reads Excel config file)
   ↓
3. Data Collection (fetches from servers)
   ├─ Progress bar shows collection status
   ├─ Logs timing and results
   └─ Can be cancelled with Ctrl+C
   ↓
4. Quality Checks (validates data consistency)
   ├─ Coherence checks (API counts vs actual data)
   ├─ Regression checks (compares with previous run)
   └─ May ask for confirmation if issues found
   ↓
5. JSON Export (writes data to files)
   ├─ endobest_inclusions.json
   └─ endobest_organizations.json
   ↓
6. Excel Export (generates workbooks)
   ├─ Reads configuration
   ├─ Applies filters/sorts
   └─ Creates Excel files
   ↓
7. Done (shows elapsed time)

--excel-only Mode Flow

1. Load existing JSON files
   ↓
2. Configuration Loading (reads Excel config)
   ↓
3. Excel Export (very fast)
   ↓
4. Done

---

What Gets Created

Output Files

JSON Files (always created):

• endobest_inclusions.json - Patient inclusion data
• endobest_organizations.json - Organization statistics

Excel Files (if configured and enabled):

• Named according to configuration (e.g., Endobest_Output_2025-01-15.xlsx)
• Multiple files if multiple workbooks configured
• Created in current directory

Log File:

• dashboard.log - Detailed execution log

File Locations

All files created in the directory where you run the script:

current_directory/
├── endobest_inclusions.json
├── endobest_organizations.json
├── endobest_organizations.json_old (backup)
├── Endobest_Output_2025-01-15.xlsx
├── Statistics_2025-01.xlsx
└── dashboard.log

---

Configuration

Configuration is done in Excel (no code changes needed):

File: config/Endobest_Dashboard_Config.xlsx

Two tables to configure Excel export:

Table 1: Excel_Workbooks

Defines WHAT files to create:

Column	Example	Meaning
workbook_name	Endobest_Output	Identifier for this workbook
template_path	templates/Endobest_Template.xlsx	Which template to use
output_filename	{workbook_name}_{extract_date_time}.xlsx	What to name the file
output_exists_action	Increment	What to do if file already exists

output_exists_action options:

• Overwrite: Replace existing file
• Increment: Create _1, _2, etc. if file exists
• Backup: Rename existing to _backup_1, etc.

Table 2: Excel_Sheets

Defines HOW to fill each sheet:

Column	Example	Meaning
workbook_name	Endobest_Output	Which workbook (must match Table 1)
sheet_name	Inclusions	Which sheet in the template
source_type	Inclusions	Where data comes from (Inclusions / Organizations / Variable)
target	DataTable	Where to put data (cell or named range)
column_mapping	{"col_id": "patient_id", "col_name": "name"}	Column definitions (JSON)
filter_condition	{"status": "active"}	Filter data (JSON)
sort_keys	[{"field": "date", "order": "asc"}]	Sort data (JSON)
value_replacement	[{"type": "bool", "true": "Yes", "false": "No"}]	Replace values (JSON)

---

Common Tasks

Task 1: Run Collection and Export

python eb_dashboard.py
# Full run: collect → checks → JSON → Excel

Task 2: Re-export Excel with New Configuration

# Edit config in Excel
python eb_dashboard.py --excel-only
# Fast! Only regenerates Excel files

Task 3: Check Data Quality

python eb_dashboard.py --check-only
# Just run quality checks on existing data

Task 4: Debug Issues

python eb_dashboard.py --debug
# Shows verbose logging
# Check dashboard.log for details

---

FAQ

Q: How long does data collection take?

A: Depends on organizations and data volume:

• Small (50 organizations): 5-10 minutes
• Medium (100+ organizations): 10-20 minutes
• Large (200+ organizations): 20-30 minutes

Q: Can I stop the script?

A: Yes, press Ctrl+C to cancel. Already-collected data is not lost.

Q: What if the script fails?

A: Check dashboard.log for error details. Most common issues:

• Network connection lost (retry)
• Invalid credentials (re-enter)
• Configuration error (fix Excel config file)

Q: Where are the Excel templates?

A: In config/templates/ folder. Configure which template to use in the Excel config file.

Q: Can I modify the Excel files after generation?

A: Yes! But running the script again may overwrite them. Use "Backup" or "Increment" action to prevent overwriting.

Q: How do I filter the data?

A: Set filter_condition in the Excel_Sheets table. Example:

{"status": "active", "visit_type": "inclusion"}

Only includes items matching ALL conditions.

Q: How do I sort the data?

A: Set sort_keys in the Excel_Sheets table. Example:

[
  {"field": "date_visit", "order": "desc"},
  {"field": "patient_name", "order": "asc"}
]

Q: What's "extract_date_time"?

A: Timestamp when data was extracted, automatically inserted into filenames:

Endobest_Output_2025-01-15T14-30-45.xlsx

Use {extract_date_time} in your output filename template.

Q: Excel formulas aren't calculating. Why?

A: Formulas only recalculate if win32com is installed. Data is correct; formulas just aren't auto-updating. To fix:

• Install: pip install pywin32
• Run script again

Q: Can I have multiple workbooks?

A: Yes! Add multiple rows to Excel_Workbooks table with different configurations.

---

Troubleshooting

"Login failed"

• Check credentials (username/password)
• Check internet connection
• Check firewall/proxy settings

"Template file not found"

• Check template file exists in config/templates/
• Check path spelling matches exactly

"Named range not found"

• Check range name in template (Excel: Formulas → Name Manager)
• Check spelling matches exactly

"No data in Excel"

• Check filter_condition isn't too restrictive
• Check filter_condition is valid JSON
• Run --check-only to verify data exists

"Excel file won't open"

• Check if file is open in another program
• Try deleting and regenerating (using --excel-only mode)
• Check dashboard.log for errors

---

Performance Tips

Slow Collection?

• Increase Number of threads (up to 20, default 12)
• Close other programs using network

Slow Excel Generation?

• Smaller datasets = faster
• Use filters to reduce data volume
• Disable formula recalculation (remove win32com requirement)

---

Support

Check Log File

# View last 50 lines
tail -n 50 dashboard.log

# View errors only
grep ERROR dashboard.log

Get Help

1. Check this document (FAQ/Troubleshooting sections)
2. Check dashboard.log for details
3. Run with --debug for more information

---

Important Notes

• Data Privacy: Patient data in JSON files - handle appropriately
• Backup Files: Script creates .json_old backups - you can delete if needed
• Configuration: Changes in Excel config take effect on next run
• Templates: Modify Excel templates in config/templates/ folder (admin only)

---

For technical details and architecture, see DOCUMENTATION_13_EXCEL_EXPORT.md