Sourcecode not visible when using PyInstaller

Important Details

How are you running Sentry?
Frozen python app created with PyInstaller

On-Premise w/ Docker, version x.y.z

Saas (sentry.io)

Other, version x.y.z, [briefly describe your environment]

Description

When my app is frozen with pyinstaller, the error context lines are not captured.

Added a couple print statements in sentry_sdk/utils.py:


def get_lines_from_file(
    filename,  # type: str
    lineno,  # type: int
    loader=None,  # type: Optional[Any]
    module=None,  # type: Optional[str]
):
    # type: (...) -> Tuple[List[Annotated[str]], Optional[Annotated[str]], List[Annotated[str]]]
    context_lines = 5
    source = None
    if loader is not None and hasattr(loader, "get_source"):
        try:
            source_str = loader.get_source(module)  # type: Optional[str]
        except (ImportError, IOError):
            source_str = None
        if source_str is not None:
            source = source_str.splitlines()

    print(f'filename: {filename}, module: {module}')
    print(f'source is None: {source is None}')
    if source is None:
        try:
            source = linecache.getlines(filename)
            print(f'source: {source}')
        except (OSError, IOError):
            return [], None, []

and at the end of sentry_sdk\client.py at line 216 in _prepare_event, i just printed print(event['exception']) to see the final event exception passed out. (trimmed some extra non imporant stuff off this)

App not frozen (working):

filename: Y:\OneDrive\Python\SMS\smseventlog\gui\gui.py, module: smseventlog.gui.gui
source is None: False

{'filename': 'smseventlog\\gui\\gui.py', 
'abs_path': 'Y:\\OneDrive\\Python\\SMS\\smseventlog\\gui\\gui.py', 
'function': 'test_error', 
'module': 'smseventlog.gui.gui', 
'lineno': 257, 
'pre_context': ['', '    def test_error(self):', '        a = 5', '        b = 6', ''], 
'context_line': '        return a / 0', 
'post_context': ['', '    def create_actions(self):', '        # Menu/shortcuts', '        t = self.active_table_widget', ''], 
'vars': {'self': '<smseventlog.gui.gui.MainWindow object at 0x000002CD9D0DFAF0>', 'a': '5', 'b': '6'}, 
'in_app': True}]}

App frozen (not working):

filename: smseventlog\gui\gui.py, module: smseventlog.gui.gui
source is None: True
source: []

 {'filename': 'smseventlog\\gui\\gui.py', 
'abs_path': 'C:\\Users\\Jayme\\Documents\\smseventlog\\dist\\smseventlog_win\\smseventlog\\gui\\gui.py', 
'function': 'test_error', 
'module': 'smseventlog.gui.gui', 
'lineno': 257,
 'pre_context': [], 
'context_line': None, 
'post_context': [], 
'vars': {'self': '<smseventlog.gui.gui.MainWindow object at 0x0000022638C4A280>', 'a': '5', 'b': '6'}, 
'in_app': True}]}}]}

the difference seems to be that in the non frozen app, the full filename is recognized Y:\OneDrive\Python\SMS\smseventlog\gui\gui.py, while in the frozen app its only recoginzed as smseventlog\gui\gui.py and the module can't be propery read to find the lines specified in the traceback.

Steps to Reproduce

What you expected to happen

Pre and post context should be captured, even in a frozen app.

Possible Solution

[If you have an idea on how this could be solved include that detail here.]

That's a known issue in the Python SDK. Moving there.

@untitaker

Thanks @untitaker, in the meantime do you know the best way to include extra data say a stacktrace or whatever else by using eg capture_exception(exception, extra_data='extra stuff here' ?

Please refer to the documentation: https://docs.sentry.io/platforms/python/enriching-error-data/additional-data/#adding-context

Any progress on this one? would be super helpful to have working thx!!! :)

@jaymegordo

Seconding @jaymegordo, having a fix/solution for this would be very beneficial :)

@jaymegordo

@jaymegordo @untitaker After reviewing the source-code, I don't think this will be very easy to handle... The file itself in a frozen executable is in relation to the directory it's in (see relevant pyinstaller docs) rather than the absolute path (which is required to grab source code lines). Additionally, even if we can find the file and open it for viewing, it's going to be a .pyc file which will not contain the same exact lines as our normal .py files we want to send stack trace information about (I suppose some de-compilation could be performed if we have the file to get at least something similar to our Python code?) I think a proper fix for this would end up being a pretty big change to the source code and how it handles this edge case.

The best I was able to come up with for the time being is to use the before_send hook (docs) in conjunction with the traceback module in Python, I'm able to do something like this:

def before_send(event, hint):
    event["extra"]["exception"] = ["".join(
        traceback.format_exception(*hint["exc_info"])
    )]
    return event

And my sdk init call:

sentry_sdk.init(
    dsn=__SENTRY_DSN__,
    release=__SENTRY_RELEASE__,
    before_send=before_send,
)

This works well enough so that I at least have some information available in the Sentry logs.

Edit The photo above is actually from a normal .py file sentry event exception being captured. Using a frozen executable results in a Sentry event more like this:

@untitaker Is this a appropriate enough solution for the issue? Or is there something else kicking about in Sentry that could handle this? I only started using Sentry the other day, so a lot of it is still new to me!

I'm surprised to see that there's any information at all reachable from within the Python process. We use our own "formatting function" on exception, I wonder where format_exception gets this extra info from?

@untitaker

@untitaker Actually, I just realized I took my screenshot from my .py file sentry event, rather than my frozen executable... The information provided in the additional data "exception" looks like this. I'll update my photo above as well.

Frozen executables do display some useful information, notably though, it removes the line that actually caused the exception being sent along in the event.

This issue has gone three weeks without activity. In another week, I will close it.

But! If you comment or otherwise update it, I will reset the clock, and if you label it Status: Backlog or Status: In Progress, I will leave it alone ... forever!

"A weed is but an unloved flower." ― Ella Wheeler Wilcox 🥀

Coming at this from a slightly different angle... In Sentry JS there's source maps, and the Sentry server can go from the compiled traceback to the original source... is there a way that approach could be applied with respect to Python .pyc content?

at build/release time, upload source .py files to sentry
distribute .pyc files to users either via pyinstaller or other mechanism
exception happens; sentry-python gets source module; line number; etc which are in the .pyc
when the error gets to the sentry server the associated source .py code is matched and displayed in the UI

Since this hasn't had any activity for a while and it's kind of low on our priority list, I've closed the issue. If someone feels like tackling it themselves feel free to reopen.

I just stumbled upon this issue, is it possible to link the source code of a python app on github to show relevant source code based on the tagged release of the code?

For example:

i have linked my github repo and created a source mapping on sentry
i created a tag on github named 1.2.3
a pyinstaller packaged app with release set to my-app@1.2.3 crashes and generates a sentry issue
as the file and line number are there, could it not pull in the source from github for that release?

i was hoping this would work out of the box, but alas im either doing something wrong or its not possible

@felixbrucker

Hey @felixbrucker, I don't know how exactly the GH code linking works since this happens outside of the SDK in Sentry itself, but I can imagine based on this thread that errors sent from a PyInstaller packaged app might be missing something we require server-side for matching to the correct GH code context. If you followed https://docs.sentry.io/product/integrations/source-code-mgmt/github/#stack-trace-linking and it doesn't work, that'd probably be the most likely reason.

I'll reopen this, but please everyone note that the priority has unfortunately not changed and we will not get to this anytime in the near future. PRs are, as always, most welcome.

Yeah, seems like it just does not work

@rcoup

Yes, this does not work right now. @rcoup idea would be the correct solution. But we will not support source maps in the Python SDK for now. Sorry.

Navigation Menu

Repository navigation

Sourcecode not visible when using PyInstaller #812

Important Details

Description

Steps to Reproduce

What you expected to happen

Possible Solution

4 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Sourcecode not visible when using PyInstaller #812

Description

Important Details

Description

Steps to Reproduce

What you expected to happen

Possible Solution

Activity

untitaker commented on Sep 2, 2020

jaymegordo commented on Sep 2, 2020

untitaker commented on Sep 2, 2020

jaymegordo commented on Oct 9, 2020

becurrie commented on Oct 27, 2020

becurrie commented on Oct 28, 2020

untitaker commented on Oct 28, 2020

becurrie commented on Oct 28, 2020

github-actions commented on Dec 23, 2021

4 remaining items

rcoup commented on Apr 26, 2022

sentrivana commented on Jul 31, 2023

felixbrucker commented on Nov 4, 2023

sentrivana commented on Nov 6, 2023

felixbrucker commented on Nov 6, 2023

antonpirker commented on Nov 6, 2023

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions