"Guid has already been taken"

Support Podmin Support
1

Hello,

I recently migrated my pod to a new server. Since then, I began to notice that comments from certain users seemed to be missing. Checking in production.log I see tons (as in, ~30k in one week) of errors with the message “Guid has already been taken”.

I logged in to an account on another pod and checked a few of the users mentioned in the error messages. They all have recent posts visible there, but nothing on my pod since the migration.

I would appreciate any help in resolving this, as it’s a pretty major problem. I’ve counted 246 unique users affected, all from external pods.

TIA

2

Here is a sample error

[2020-06-14T00:13:59] ERROR PID-338 TID-47371766023560 DiasporaFederation::Federation::Receiver: failed to receive public message: DiasporaFederation::Discovery::DiscoveryError: Failed discovery for xxxxx@venera.social: ActiveRecord::RecordInvalid: Validation failed: Guid has already been taken
3

And in case it’s relevant, the database is postgres 11.

4

This is really strange … so it’s trying to fetch profiles but then complains that the GUIDs already exist? Have you checked if the GUIDs really already exist and if so, to which other profile/pod they belong?

On diaspora you can get the GUID of a user it tried to fetch by going to this URL: https://venera.social/.well-known/webfinger?resource=acct:xxxxx@venera.social which should then respond with a JSON and there should be an element with rel: "http://microformats.org/profile/hcard", and the href of this element, contains already the GUID as last part.

But it looks like the profile that causes issues is from a friendica server, and there it’s a bit more complicated, because you actually need to go to the hCard and then in the html source there should be an element class="entity_uid" which contains the GUID.

Then you can check in your database if there already is a person with that GUID, and for which profile it is, and if all are from the same pod or what’s going on.

But without knowing which profiles/GUIDs/pods are affected there is not much I can do to help you. This should never happen since GUIDs should be unique, but maybe a pod tried to change their domain (which is not supported) and now all users are duplicated with the same GUID, but since it’s a new domain, your pod doesn’t know them with the new domain and tries to fetch them, but the GUID is still used by the same user from the old domain …

5

I can observe similar patterns in my production log, albeit super low traffic overall. I have no cases from venera.social, but one from another Friendica instance:

DiasporaFederation::Discovery::DiscoveryError: Failed discovery for silverwizard@convenient.email: ActiveRecord::RecordInvalid: Validation failed: Guid has already been taken, Person with same GUID already exists: silverwizard@friendica.obscuritus.ca

On further investigation:

# curl -I https://friendica.obscuritus.ca
HTTP/1.1 302 Found
Location: https://convenient.email/profile/silverwizard

So yeah, this is Friendica people forcefully renaming their instances and profiles. As such, diaspora* refusing to accept anything from their new domain is not a bug, it’s a feature - it’s effectively preventing identity theft.

1 Like
6

FWIW, we are very explicit in our config comment that the URL should not be changed after initial setup. I do not know if Friendica is doing the same, but I surely hope so. /cc @michaelvogel

7

This is not just a Friendica issue. There are errors from diasp.org, framasphere.org, pluspora.com, etc. There doesn’t seem to be a pattern that I can discern, but I would strongly guess that the problem is on my end.

I’ll see if I can track down one or two of those guids. Which table should I be looking in?

Thanks

8

So, I picked a user from one of the errors, who’s from diasp.org. This is a long-time D* user who I know, so it’s definitely not a weird or wonky account, or a Friendica instance. I grabbed the guid from the hcard and looked it up in the people table, and the diaspora_handle matches the user the error complains about.

9

So if you run RAILS_ENV=production bin/rails c and then run Person.by_account_identifier("user@pod") in it does it return the person? Or does it also throw the error?

Does the diaspora_handle in your database somehow contain uppercase handles? Because diaspora searches them all lowercase. Otherwise I don’t see how it should try to fetch a profile that’s already known.

10 
2.4.9 :002 > Person.by_account_identifier("xxxxx@diasp.org")
 => nil

Yes, all lowercase.

11

And how is the diaspora_handle stored in the database?

12

It’s all lowercase…

13

Or are there spaces somewhere at the beginning or end?

So the code does this: Person.find_by(diaspora_handle: diaspora_id.strip.downcase).

So if it is uppercase or has spaces at the beginning or end in the database, it wouldn’t find it, but if it’s all lowercase it should find it, if it exists.

14

Oh wait, it also tries to refetch the person if the profile is missing, but the person exists … did you, at some point, manually delete profiles in your database?

15

Nope. All I did was migrate to a new server by rsyncing /var/lib/postgresql from the old serve to the new one. If I delete the user from people will it re-fetch them?

16

I still don’t understand how your database doesn’t find a person with the diaspora_handle, but you say it exists with the exact same handle?

Does it return the person when you select it directly on the database? Or when using Person.find_by(diaspora_handle: "user@pod")

And sorry, I mixed stuff up above, if it’s the missing profile-problem, by_account_identifier wouldn’t return nil. And you shouldn’t delete stuff manually in the DB, that usually only causes more problems.

17

Sorry, I must have done something wrong last time. I just tried again

diaspora@5ce5d225b6da:~/diaspora$ RAILS_ENV=production bin/rails c
Person.by_account_identifier("xxxx@diasp.org")
Loading production environment (Rails 5.2.4.1)
2.4.9 :001 > Person.by_account_identifier("xxxx@diasp.org")
 => #<Person id: 183, guid: "9c54ab737368bed2", diaspora_handle: "xxxx@diasp.org", 
...
18 
diaspora_production=# select * from people where guid = '9c54ab737368bed2';
 id  |       guid       |   diaspora_handle    |                      serialized_public_key                       | owner_id |     created_at      |     updated_at      | closed_account | fetch_status | pod_id 
-----+------------------+----------------------+------------------------------------------------------------------+----------+---------------------+---------------------+----------------+--------------+--------
 183 | 9c54ab737368bed2 | xxxx@diasp.org | -----BEGIN RSA PUBLIC KEY-----                                  +|          | 2017-03-06 00:25:58 | 2017-03-06 00:25:59 | f              |            4 |     31
     |                  |                      |
19

Does that person have a profile?

In the Rails console,

Person.by_account_identifier("foo@exmaple.com").profile

and in a pg-shell

SELECT * FROM profiles WHERE person_id = 183;
20

Yes, both method return results.