Skip to content
This repository was archived by the owner on Jan 23, 2023. It is now read-only.
This repository was archived by the owner on Jan 23, 2023. It is now read-only.

Handling of withheld accounts and/ or tweets #428

Closed
Closed
Handling of withheld accounts and/ or tweets#428
Labels
enhancementNew feature or request

Description

@schmittlauch
schmittlauch
opened on Mar 4, 2022

Describe the bug
Continuation from Twitter:
Issue 1: When encountering tweets withheld in the locality where the users currently resides in (or maybe based on Twitter location settings, I do not know), the warning text is displayed in the post body – but the link as found in the WebUI is not clickable.

Issue 2: In case a hole account is withheld, the tweets do load successfulli-ysh (all show the same withheld text, but at least they are displayed) in their profile, but their profile description fails to load. Instead, the gear loading animation keeps on spinning forever.

To Reproduce
Steps to reproduce the behavior:

  1. run the normal binary with active debug outputs: G_MESSAGES_DEBUG=cawbird cawbird
  2. From an EU country, search for a withheld account like de_rt_com
  3. Click on their profile in the search results
  4. See the issues appear. Log after clicking on the profile: see below
(cawbird:245994): cawbird-DEBUG: 20:24:23.849: REST: GET 1.1/statuses/user_timeline.json
(cawbird:245994): cawbird-DEBUG: 20:24:23.849: REST: GET 1.1/statuses/user_timeline.json
(cawbird:245994): cawbird-DEBUG: 20:24:23.849: REST: GET 1.1/friendships/show.json
(cawbird:245994): cawbird-DEBUG: 20:24:23.849: REST: GET 1.1/users/show.json
(cawbird:245994): cawbird-DEBUG: 20:24:23.849: REST: GET 1.1/statuses/user_timeline.json
cawbird-INFO: 20:24:24.157: Skipping entity - expected https://support.twitter.com/articles/20169222 but found Learn more. Likely bad indices (323 to 333)
cawbird-INFO: 20:24:24.160: Skipping entity - expected https://support.twitter.com/articles/20169222 but found Learn more. Likely bad indices (323 to 333)
cawbird-INFO: 20:24:24.161: Skipping entity - expected https://support.twitter.com/articles/20169222 but found Learn more. Likely bad indices (323 to 333)
cawbird-INFO: 20:24:24.163: Skipping entity - expected https://support.twitter.com/articles/20169222 but found Learn more. Likely bad indices (323 to 333)
cawbird-INFO: 20:24:24.164: Skipping entity - expected https://support.twitter.com/articles/20169222 but found Learn more. Likely bad indices (323 to 333)
cawbird-INFO: 20:24:24.167: Skipping entity - expected https://support.twitter.com/articles/20169222 but found Learn more. Likely bad indices (323 to 333)
cawbird-INFO: 20:24:24.168: Skipping entity - expected https://support.twitter.com/articles/20169222 but found Learn more. Likely bad indices (323 to 333)
cawbird-INFO: 20:24:24.171: Skipping entity - expected https://support.twitter.com/articles/20169222 but found Learn more. Likely bad indices (323 to 333)
cawbird-INFO: 20:24:24.172: Skipping entity - expected https://support.twitter.com/articles/20169222 but found Learn more. Likely bad indices (323 to 333)
cawbird-INFO: 20:24:24.176: Skipping entity - expected https://support.twitter.com/articles/20169222 but found Learn more. Likely bad indices (323 to 333)
cawbird-INFO: 20:24:24.177: Skipping entity - expected https://support.twitter.com/articles/20169222 but found Learn more. Likely bad indices (323 to 333)
cawbird-INFO: 20:24:24.180: Skipping entity - expected https://support.twitter.com/articles/20169222 but found Learn more. Likely bad indices (323 to 333)
cawbird-INFO: 20:24:24.182: Skipping entity - expected https://support.twitter.com/articles/20169222 but found Learn more. Likely bad indices (323 to 333)
cawbird-INFO: 20:24:24.186: Skipping entity - expected https://support.twitter.com/articles/20169222 but found Learn more. Likely bad indices (323 to 333)
cawbird-INFO: 20:24:24.187: Skipping entity - expected https://support.twitter.com/articles/20169222 but found Learn more. Likely bad indices (323 to 333)
cawbird-INFO: 20:24:24.190: Skipping entity - expected https://support.twitter.com/articles/20169222 but found Learn more. Likely bad indices (323 to 333)
cawbird-INFO: 20:24:24.191: Skipping entity - expected https://support.twitter.com/articles/20169222 but found Learn more. Likely bad indices (323 to 333)
cawbird-INFO: 20:24:24.193: Skipping entity - expected https://support.twitter.com/articles/20169222 but found Learn more. Likely bad indices (323 to 333)
cawbird-INFO: 20:24:24.194: Skipping entity - expected https://support.twitter.com/articles/20169222 but found Learn more. Likely bad indices (323 to 333)
cawbird-INFO: 20:24:24.196: Skipping entity - expected https://support.twitter.com/articles/20169222 but found Learn more. Likely bad indices (323 to 333)
cawbird-INFO: 20:24:24.324: Skipping entity - expected https://support.twitter.com/articles/20169222 but found Learn more. Likely bad indices (323 to 333)
cawbird-INFO: 20:24:24.324: Skipping entity - expected https://support.twitter.com/articles/20169222 but found Learn more. Likely bad indices (323 to 333)
cawbird-INFO: 20:24:24.324: Skipping entity - expected https://support.twitter.com/articles/20169222 but found Learn more. Likely bad indices (323 to 333)
cawbird-INFO: 20:24:24.324: Skipping entity - expected https://support.twitter.com/articles/20169222 but found Learn more. Likely bad indices (323 to 333)
cawbird-INFO: 20:24:24.324: Skipping entity - expected https://support.twitter.com/articles/20169222 but found Learn more. Likely bad indices (323 to 333)
cawbird-INFO: 20:24:24.324: Skipping entity - expected https://support.twitter.com/articles/20169222 but found Learn more. Likely bad indices (323 to 333)
cawbird-INFO: 20:24:24.324: Skipping entity - expected https://support.twitter.com/articles/20169222 but found Learn more. Likely bad indices (323 to 333)
cawbird-INFO: 20:24:24.324: Skipping entity - expected https://support.twitter.com/articles/20169222 but found Learn more. Likely bad indices (323 to 333)
cawbird-INFO: 20:24:24.324: Skipping entity - expected https://support.twitter.com/articles/20169222 but found Learn more. Likely bad indices (323 to 333)
cawbird-INFO: 20:24:24.324: Skipping entity - expected https://support.twitter.com/articles/20169222 but found Learn more. Likely bad indices (323 to 333)

(cawbird:245994): libsoup-CRITICAL **: 20:24:24.512: soup_session_queue_message: assertion 'SOUP_IS_MESSAGE (msg)' failed
cawbird-INFO: 20:24:24.513: Skipping entity - expected https://support.twitter.com/articles/20169222 but found Learn more. Likely bad indices (323 to 333)
cawbird-INFO: 20:24:24.652: Skipping entity - expected https://support.twitter.com/articles/20169222 but found Learn more. Likely bad indices (323 to 333)
cawbird-INFO: 20:24:24.652: Skipping entity - expected https://support.twitter.com/articles/20169222 but found Learn more. Likely bad indices (323 to 333)
cawbird-INFO: 20:24:24.652: Skipping entity - expected https://support.twitter.com/articles/20169222 but found Learn more. Likely bad indices (323 to 333)
cawbird-INFO: 20:24:24.652: Skipping entity - expected https://support.twitter.com/articles/20169222 but found Learn more. Likely bad indices (323 to 333)
cawbird-INFO: 20:24:24.652: Skipping entity - expected https://support.twitter.com/articles/20169222 but found Learn more. Likely bad indices (323 to 333)
cawbird-INFO: 20:24:24.653: Skipping entity - expected https://support.twitter.com/articles/20169222 but found Learn more. Likely bad indices (323 to 333)
cawbird-INFO: 20:24:24.653: Skipping entity - expected https://support.twitter.com/articles/20169222 but found Learn more. Likely bad indices (323 to 333)
cawbird-INFO: 20:24:24.653: Skipping entity - expected https://support.twitter.com/articles/20169222 but found Learn more. Likely bad indices (323 to 333)
cawbird-INFO: 20:24:24.653: Skipping entity - expected https://support.twitter.com/articles/20169222 but found Learn more. Likely bad indices (323 to 333)
cawbird-INFO: 20:24:24.653: Skipping entity - expected https://support.twitter.com/articles/20169222 but found Learn more. Likely bad indices (323 to 333)

System details:

  • OS: NixOS-21.11
  • Version: 1.4.2
  • Package type: distro

Activity

  1. IBBoard commented on Mar 4, 2022

    @IBBoard
    IBBoard
    on Mar 4, 2022
    Owner

    So, it is our sanity checking that breaks the links!

    If someone can run a debug build from the command-line, click a tweet to focus it, and then press 'k' (shortcuts) and dump the JSON for the tweet here then that'd be great.

    I'll try to work out what we can do about the header and account info. libsoup-CRITICAL **: 20:24:24.512: soup_session_queue_message: assertion 'SOUP_IS_MESSAGE (msg)' failed suggests that something might have happened that we could detect and handle. But I don't know why it wouldn't be a SOUP message. I can see it might not be JSON, but an invalid SOUP message is odd.

  2. IBBoard
    added
    enhancementNew feature or request
    info requiredThis doesn't seem right or more details are required
    on Mar 4, 2022

  3. IBBoard commented on Mar 4, 2022

    @IBBoard
    IBBoard
    on Mar 4, 2022
    Owner

    If no-one is able to provide the JSON then I might be able guess at it. We're currently checking that the unformatted text matches the "original text", which links define as the URL (which is a t·co URL). But "Learn More" isn't a URL. So I can try comparing against the display text as well (which won't match for normal links, because the display text is the truncated version, but it may work for these links).

    I might also be able to do something with Twitter profiles and locations and pretending I'm German ðŸĪ”

  4. IBBoard commented on Mar 4, 2022

    @IBBoard
    IBBoard
    on Mar 4, 2022
    Owner

    Setting my country to Germany worked 🙂 And when I did, it decided to ask lots of privacy questions about what to enable and disable. Almost as if Europe has better GDPR protection/enforcement than the UK 🧐

    {
  "created_at" : "Fri Mar 04 20:17:19 +0000 2022",
  "id" : 1499841440712732677,
  "id_str" : "1499841440712732677",
  "full_text" : "@de_rt_com's account has been withheld in Austria, Belgium, Bulgaria, Croatia, Cyprus, Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Ireland, Italy, Latvia, Lithuania, Luxembourg, Malta, Netherlands, Poland, Portugal, Romania, Slovakia, Slovenia, Spain, Sweden in response to a legal demand. Learn more.",
  "truncated" : false,
  "display_text_range" : [
    0,
    334
  ],
  "entities" : {
    "hashtags" : [],
    "symbols" : [],
    "user_mentions" : [],
    "urls" : [
      {
        "url" : "https://support.twitter.com/articles/20169222",
        "expanded_url" : "https://support.twitter.com/articles/20169222",
        "display_url" : "Learn more",
        "indices" : [
          323,
          333
        ]
      }
    ]
  },
  "source" : "<a href=\"https://mobile.twitter.com\" rel=\"nofollow\">Twitter Web App</a>",
  "in_reply_to_status_id" : null,
  "in_reply_to_status_id_str" : null,
  "in_reply_to_user_id" : null,
  "in_reply_to_user_id_str" : null,
  "in_reply_to_screen_name" : null,
  "user" : {
    "id" : 2561586612,
    "id_str" : "2561586612",
    "name" : "",
    "screen_name" : "de_rt_com",
    "location" : "",
    "description" : "@de_rt_com's account has been withheld in Austria, Belgium, Bulgaria, Croatia, Cyprus, Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Ireland, Italy, Latvia, Lithuania, Luxembourg, Malta, Netherlands, Poland, Portugal, Romania, Slovakia, Slovenia, Spain, Sweden in response to a legal demand. Learn more.",
    "url" : null,
    "entities" : {
      "description" : {
        "urls" : [
          {
            "url" : "https://support.twitter.com/articles/20169222",
            "expanded_url" : "https://support.twitter.com/articles/20169222",
            "display_url" : "Learn more",
            "indices" : [
              323,
              333
            ]
          }
        ]
      }
    },
    "protected" : false,
    "followers_count" : 51484,
    "friends_count" : 473,
    "listed_count" : 634,
    "created_at" : "Wed Jun 11 15:54:00 +0000 2014",
    "favourites_count" : 2539,
    "utc_offset" : null,
    "time_zone" : null,
    "geo_enabled" : false,
    "verified" : false,
    "statuses_count" : 104627,
    "lang" : null,
    "contributors_enabled" : false,
    "is_translator" : false,
    "is_translation_enabled" : false,
    "profile_background_color" : "000000",
    "profile_background_image_url" : "http://abs.twimg.com/images/themes/theme1/bg.png",
    "profile_background_image_url_https" : "https://abs.twimg.com/images/themes/theme1/bg.png",
    "profile_background_tile" : false,
    "profile_image_url" : "",
    "profile_image_url_https" : "",
    "profile_image_extensions_alt_text" : null,
    "profile_banner_extensions_alt_text" : null,
    "profile_link_color" : "6DC331",
    "profile_sidebar_border_color" : "000000",
    "profile_sidebar_fill_color" : "000000",
    "profile_text_color" : "000000",
    "profile_use_background_image" : false,
    "has_extended_profile" : false,
    "default_profile" : false,
    "default_profile_image" : false,
    "following" : false,
    "follow_request_sent" : false,
    "notifications" : false,
    "translator_type" : "none",
    "withheld_scope" : "user",
    "withheld_in_countries" : [
      "AT",
      "BE",
      "BG",
      "CY",
      "CZ",
      "DE",
      "DK",
      "EE",
      "ES",
      "FI",
      "FR",
      "GR",
      "HR",
      "HU",
      "IE",
      "IT",
      "LT",
      "LU",
      "LV",
      "MT",
      "NL",
      "PL",
      "PT",
      "RO",
      "SE",
      "SI",
      "SK"
    ]
  },
  "geo" : null,
  "coordinates" : null,
  "place" : null,
  "contributors" : null,
  "withheld_scope" : "user",
  "withheld_in_countries" : [
    "AT",
    "BE",
    "BG",
    "CY",
    "CZ",
    "DE",
    "DK",
    "EE",
    "ES",
    "FI",
    "FR",
    "GR",
    "HR",
    "HU",
    "IE",
    "IT",
    "LT",
    "LU",
    "LV",
    "MT",
    "NL",
    "PL",
    "PT",
    "RO",
    "SE",
    "SI",
    "SK"
  ],
  "is_quote_status" : false,
  "retweet_count" : 1,
  "favorite_count" : 2,
  "favorited" : false,
  "retweeted" : false,
  "possibly_sensitive" : false,
  "lang" : "de"
}
  5. IBBoard
    added a commit that references this issue on Mar 4, 2022

    Re #428: Fix "Learn More" link on witheld tweet

    53a37bc

  6. IBBoard commented on Mar 4, 2022

    @IBBoard
    IBBoard
    on Mar 4, 2022
    Owner

    Got most of a fix for the rest of the profile issues. Just need to iron out some resource issues. And make our default avatar for these situations look less happy.

  7. schmittlauch commented on Mar 4, 2022

    @schmittlauch
    schmittlauch
    on Mar 4, 2022
    ContributorAuthor

    Ah great, I just had it figured out how to do a debug build ^^

  8. IBBoard
    removed
    info requiredThis doesn't seem right or more details are required
    on Mar 5, 2022
  9. IBBoard
    added 5 commits that reference this issue on Mar 5, 2022

    Re #428: Handle withheld accounts having null avatar URLs

    6d9c4a0

    Re #428: Add specific "withheld" avatar

    32a5e19

    Re #428: Update i18n for "Withheld account"

    45b3326

    Re #428: Avoid JSON `...with_default` method

    8babd5f

    Re #428: Reset `is_withheld` each reload

    4cc3385

  10. IBBoard commented on Mar 16, 2022

    @IBBoard
    IBBoard
    on Mar 16, 2022
    Owner

    I'm going to assume that this is working. We should release it soon.

  11. IBBoard
    closed this as completedon Mar 16, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions