python open function trunctating output

Question

I am very new to python and coding in general, so any help or point in the right direction is greatly appreciated.

I am trying to print an output to a file using python's open function. when I print the 'jstat_output' variable to the terminal it appears fine, but when I try to read the textfile it has been truncated.

the code:

with open(path_to_file, 'w', newline='') as o:
for item in jstat_output:
    o.write(f'{item}\n')

terminal output:

[Container(PayloadLength=81, Payload=u'  S0     S1     E      O      M     CCS    YGC     YGCT    FGC    FGCT     GCT   '), Container(PayloadLength=81, Payload=u'  S0     S1     E      O      M     CCS    YGC     YGCT    FGC    FGCT     GCT   ')]

notepad viewer:

Container: PayloadLength = 81 Payload = u' S0 S1 E O M '... (truncated, total 81) Container: PayloadLength = 81 Payload = u' S0 S1 E O M '... (truncated, total 81)

if anyone knows why this happens, or perhaps a different method to write to a file it would be a massive help; thanks.

    for pid in extracted_numbers:
        exec_resp = ecs.execute_command(
            cluster=f'{clusterArn[0]}',
            container=f'{containerNames[ttype]}',
            task=f'{tArns[tarn]}',
            command=f'jstat -gcutil {pid} 250',
            interactive=True
    )
    session = exec_resp['session']
    connection = websocket.create_connection(session['streamUrl'])
    try:
        init_payload = {
            "MessageSchemaVersion": "1.0",
            "RequestId": str(uuid.uuid4()),
            "TokenValue": session['tokenValue']
        }
        connection.send(json.dumps(init_payload))

        AgentMessageHeader = c.Struct(
            'HeaderLength' / c.Int32ub,
            'MessageType' / c.PaddedString(32, 'ascii'),
        )

        AgentMessagePayload = c.Struct(
            'PayloadLength' / c.Int32ub,
            'Payload' / c.PaddedString(c.this.PayloadLength, 'ascii')
        )

        while True:
            response = connection.recv()

            message = AgentMessageHeader.parse(response)

            if 'channel_closed' in message.MessageType:
                raise Exception('Channel closed before command output was received')

            if 'output_stream_data' in message.MessageType:
                break
    finally:
        connection.close()

    payload_message = AgentMessagePayload.parse(response[message.HeaderLength:])
    jstat_output.append(payload_message)

print(jstat_output)
with open(path_to_file, openvar, newline='') as o:
    print(f'{jstat_output}', file=o)

this is the portion of the code that is related to the problem.

The code you've shared doesn't print anything. Can you include the part where you do the printing? — Samwise
– Samwise, Commented Jul 3, 2022 at 14:08
through trial and error, i've come to see that it has something to do with the for loop. when i remove the for loop it works. i'd like to keep the for loop so if anyone knows how to accomplish this i'd really be grateful for the help. — yaniv
– yaniv, Commented Jul 3, 2022 at 14:10
It looks like the terminal output doesn't match the default string representation of Container so I expect you're doing something extra with your print() call to make it print the full contents. Seeing that code would help. Seeing the definition of Container might also suggest an easier solution. — Samwise
– Samwise, Commented Jul 3, 2022 at 14:11

Cardstdani · Accepted Answer · 2022-07-03 14:27:43Z

2

The reason that your print() prints the non-truncated version is that it's printing the repr() of each item instead of the str() of it.

I believe this will do what you want:

with open(path_to_file, 'w') as o:
    for item in jstat_output:
        o.write(repr(item) + "\n")

The clue is that jstat_output is a list and when you printed it with:

print(jstat_output)

you got the desired output.

When a list is rendered as a string, it renders the repr() of each of its individual items. If the items are formatted individually (e.g. by doing f"{item}" or print(item)), you're instead getting their str() representation. In this case, the str() representation is truncated when the item is large, whereas repr() includes the whole thing. Explicitly calling repr(item) instead of using an f-string should therefore get you the output you're looking for. You could also still use an f-string as long as you put the repr() inside the {}, i.e. f"{repr(item)}\n".

A better solution might be to write your own function to output a Container in the desired format rather than relying on its __repr__ implementation, but that would require being able to see the definition of the Container class.

edited Jul 3, 2022 at 14:27

Cardstdani

5,2513 gold badges14 silver badges37 bronze badges

answered Jul 3, 2022 at 14:13

Samwise

72.1k3 gold badges36 silver badges52 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Kelly Bundy Over a year ago

Or f"{item!r}".

yaniv Over a year ago

this worked! thank you, I didn't know that rendering a list as a string would create this issue.

Collectives™ on Stack Overflow

python open function trunctating output

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related