This is a work in progress. I’ll keep updating it until I stop updating it.

Technical Best Practices

Don’t Create Many Projects

Do not create one project for each notebook.

I know it’s tempting. Don’t do it. You’ll end up with dozens of notebooks, and BQuant has no tools to organize them—no folders, no tags, etc. Opening each notebook takes time (it’s slow).

Instead, create one main project for most of your work. Organize the code inside the project.

Keep it simple.

Custom Environments is a balancing act

Custom Environments are necessary for Jupyter Extensions and suitable for packages that change infrequently. However, they take minutes to start after each change, which can be annoying when testing or trying new versions of things. We use custom environments for our “production” applications. We’ll keep a minimum Custom Environment for development and load what we need at runtime. We’re still finding the right balance here.

Multiple Browser Tabs

If you copy/paste the BQuant URL into a separate browser tab, it will open another window to the same Project session. Browser tabs are often easier to work with than Jupyter tabs, plus you can arrange them as regular windows.

If you use multiple Tabs, it helps to also disable Chrome’s ProcessPerSite.

Use Git/Github

Keep your code / projects in Git (via Github or a private Git repository). BQuant’s built-in versioning and sharing are relatively primitive.

Github Personal Access Token

If using Github, you can set a Personal Access Token for each user as an Environment Variable. Then, clone with the token: !git clone https://<token>@github.com/orgname/repo

nbstrip ipynb files

We use the nbstrip pre-commit to commit code-only notebooks. This strips any data from the notebook.

Map out your data sources

BQuant Enterprise is tightly locked down*, so any systems you need must be individually requested.

Therefore, start talking with the Bloomberg rep early about the data sources you’ll need access to, such as cloud (ie: S3) and internal (on-prem) IT resources.

* This is a good thing: starting “locked down” and gradually opening up is better.

Use the BQuant Help Center

This is relatively new and took me a while to appreciate. The field search is probably better than FLDS<go> although I still use FLDS a lot as I can compare different field data sources.

Terminal Commands

• bqlx<go>: BQL in Excel Help + some useful documents
• flds<go>: Field Search
• ni blpbql<go>: News relating to BQL

syncignore

Create a temp directory at the root of your project, and create a syncignore (no extension) file in the root of your project. In the syncignore file, put the word: temp. This will prevent the contents of the temp directory from synchronizing.

Use the temp directory for transient files that shouldn’t be synchronized. BQuant projects have a size limit, and while S3 should be where you store data, you’ll commonly run into cases with large temporary files.

* This is an undocumented feature, use at your own risk

BQL Tips

Get Good at BQL

BQL is simple. Learning how to write queries will save you time in the future.

Use the BQL Query Strings

BQL query format is a matter of personal preference. We choose to use BQL text queries and not the BQL object model.

Use This:

request = """
get(
    px_high
)
for(
    ['IBM US Equity']
)
"""

Not This (Object Model):

security = "IBM US Equity"
high = bq.data.px_high()
request = bql.Request(security, high)
response = bq.execute(request)

Doublecheck the Data

Check your data against Terminal, especially when you’re just starting out with BQL. It’s easy to write a query that differs from what you get in Terminal. Most of the time, there’s a good explanation. Sometimes there’s a terrible explanation. Either way: better to understand the differences early rather than later when the CIO is wondering why the information doesn’t match what they see.

There’s some subtle differences.. such as how BDP and BQL handle LTM values for 4Q data. BQL returns the annual result, whereas BDP returns the sum of previous four quarters. Using FPTS=Q (TODO: Check), you can make BQL behave like BDP, if that’s what you want.

Other Topics

There are a number of other topics to consider:

• Scheduling: BQuant provides some built-in scheduling methods, so you can run tasks and run background processes.
• Caching: BQuant provides an S3 bucket for each user to read/write. This is handy for caching models and expensive queries.
• Test Automation: Testing is a bit difficult. Setting up clear test cases early is helpful at avoid the difficult task of dealing with tests later.
• CI/CD: There’s several ways to build into BQuant. Our preferred method is through GitHub Actions writing to an S3 bucket that BQuant has read-only access to.
• Observability: Since BQuant is a locked-down environment, and every org is different, a conversation with your Bloomberg team is needed to determine what methods you can use to send notifications to users. Yes, I’m punting on this, it’s complicated and depends a lot on how your org works today.;

Wish List

Some features or improvements I’d love to see:

• Organizing projects: A single flat list of projects gets complicated quickly
• Projects directly from Git: Since we follow a git-centric workflow, I’d love to create a project directly from a Git repo, and have a fully stateless project.
• Shared ownership of custom environment: A single individual owns custom environments today: this gets difficult to manage, with all changes going through one person.
• s3fs-fuse: BQuant provides a number of S3 buckets; being able to mount the S3 buckets to a top-level project folder would be nice.
• VSCode remote support: I prefer doing my heavy coding in VSCode.