بررسی دقیق نشر اسناد گوگل (قسمت دوم)

Attributes_11

creationDate (type: integer(), default: nil) – used for history – the first and last time we have seen this anchor. creation_date also used for Freshdocs Twitter indexing, a retweet is an anchor of the original tweet. This field records the time when a retweet is created.

origText (type: String.t, default: nil) – Original text, including capitalization and punctuation. Runs of whitespace are collapsed into a single space.

context2 (type: integer(), default: nil) – This is a hash of terms near the anchor. (This is a second-generation hash replacing the value stored in the ‘context’ field.)

fontsize (type: integer(), default: nil) –

experimental (type: boolean(), default: nil) – If true, the anchor is for experimental purposes and should not be used in serving.

fragment (type: String.t, default: nil) – The URL fragment for this anchor (the foo in http://www.google.com#foo)

sourceType (type: integer(), default: nil) – is to record the quality of the anchor’s source page and is correlated with but not identical to the index tier of the source page. In the docjoins built by the indexing pipeline (Alexandria), – Anchors marked TYPE_HIGH_QUALITY are from base documents. – Anchors marked TYPE_MEDIUM_QUALITY are from documents of medium quality (roughly but not exactly supplemental tier documents). – Anchors marked TYPE_LOW_QUALITY are from documents of low quality (roughly but not exactly blackhole documents). Note that the source_type can also be used as an importance indicator of an anchor (a lower source_type value indicates a more important anchor), so it is important to enforce that TYPE_HIGH_QUALITY < TYPE_MEDIUM_QUALITY < TYPE_LOW_QUALITY To add a new source type in future, please maintain the proper relationship among the types as well. TYPE_FRESHDOCS, only available in freshdocs indexing, is a special case and is considered the same type as TYPE_HIGH_QUALITY for the purpose of anchor importance in duplicate anchor removal.

pagerankWeight (type: number(), default: nil) – Weight to be stored in linkmaps for pageranker

isLocal (type: boolean(), default: nil) – The bit ~roughly~ indicates whether an anchor’s source and target pages are on the same domain. Note: this plays no role in determining whether an anchor is onsite, ondomain, or offdomain in mustang (i.e., the bit above).

originalTargetDocid (type: String.t, default: nil) – The docid of the anchor’s original target. This field is available if and only if the anchor is forwarded.

fullLeftContext (type: list(String.t), default: nil) – The full context. These are not written out in the linklogs.

expired (type: boolean(), default: nil) – true iff exp domain

catfishTags (type: list(integer()), default: nil) – CATfish tags attached to a link. These are similar to link tags, except the values are created on the fly within Cookbook. See: http://sites/cookbook/exporting/indexing

deletionDate (type: integer(), default: nil) –

linkTags (type: list(integer()), default: nil) – Contains info on link type, source page, etc.

forwardingTypes (type: integer(), default: nil) – How the anchor is forwarded to the canonical, available only for forwarded anchors (i.e., the field is set). The forwarding types are defined in URLForwardingUtil (segindexer/segment-indexer-util.h). Always use URLForwardingUtil to access this field and use URLForwardingUtil::GetAnchorForwardingReason to get the explanation how the anchor is forwarded to the canonical. NOTE: Use with caution as it is only set for docjoins generated using the urlmap from repository/updater.

possiblyOldFirstseenDate (type: boolean(), default: nil) – DEPRECATED. It used to be set if firstseen_date is not set. It’s to indicate that the anchor is possibly old, but we don’t have enough information to tell until the linkage map is updated. TODO(hxu) rename it to possibly_old_firstseen_date_DEPRECATED after clean up other dependencies.

locality (type: integer(), default: nil) – For ranking purposes, the quality of an anchor is measured by its “locality” and “bucket”. See quality/anchors/definitions.h for more information.

demotionreason (type: integer(), default: nil) – DEPRECATED

parallelLinks (type: integer(), default: nil) – The number of additional links from the same source page to the same target domain. Not populated if is_local is true.

text (type: String.t, default: nil) – Space-delimited anchor words. Text that needs segmentation (like CJK or Thai) is unsegmented, since we set FLAGS_segment_during_lexing to false in mr-linkextractor.cc .

source (type: GoogleApi.ContentWarehouse.V1.Model.AnchorsAnchorSource.t, default: nil) –

bucket (type: integer(), default: nil) –

fullRightContext (type: list(String.t), default: nil) –

targetUrlEncoding (type: integer(), default: nil) – A given target URL may be found in different encodings in different documents. We store the URL encoding with each source anchor so that we can count them later to find the encoding most likely to be expected by the Web site. Around 0.7% of target URLs are expected to require a non-default value here. The default value 0 is referenced in C++ as webutil::kDefaultUrlEncoding. See also webutil/urlencoding.

compressedOriginalTargetUrl (type: String.t, default: nil) – The anchor’s original target url, compressed. Available only in Alexandria docjoins when the anchor is forwarded.

firstseenDate (type: integer(), default: nil) – # days past Dec 31, 1994, 23:00:00 UTC (Unix time @788914800) that this link was first seen. Should never occupy more than 15 bits. NOTE: this is NOT the same as creation_date; firstseen_date is filled during link extraction

setiPagerankWeight (type: number(), default: nil) – TEMPORARY

context (type: integer(), default: nil) –

linkAdditionalInfo (type: GoogleApi.ContentWarehouse.V1.Model.Proto2BridgeMessageSet.t, default: nil) – Additional information related to the anchor, such as additional anchor text or scores.

type (type: integer(), default: nil) – DEPRECATED: Now in link_tags

firstseenNearCreation (type: boolean(), default: nil) – true if we think ‘firstseen_date’ is an accurate estimate of when the link was actually added to the source page. false if it may have existed for some time before we saw it.

lastUpdateTimestamp (type: integer(), default: nil) – Used for history and freshness tracking – the timestamp this anchor is updated in indexing.

offset (type: integer(), default: nil) – This is the offset for the first term in the anchor – it can be used as a unique ID for the anchor within the document and compared against all per-tag data. This is measured in bytes from the start of the document. We write this out to the linklogs to recover the original order of links after source/target forwarding. This is necessary for computing the global related data.

weight (type: integer(), default: nil) – weights are 0-127

deleted (type: boolean(), default: nil) –

encodedNewsAnchorData (type: integer(), default: nil) – Encoded data containing information about newsiness of anchor. Populated only if anchor is classified as coming from a newsy, high quality site. Encoded data for anchor sources are being stored in googledata/quality/freshness/news_anchors/encoded_news_anchors_data.txt Scores are being computed with quality/freshness/news_anchors/ routines.

compressedImageUrls (type: list(String.t), default: nil) – If the anchor contained images, these image urls are stored here in compressed form.

timestamp (type: String.t, default: nil) – This field is DEPRECATED and no longer filled. For source page crawl timestamp, use Source.crawl_timestamp. Next tag id should be 62.

GoogleApi.ContentWarehouse.V1.Model.AnchorsAnchorSource

attributes of the source document for the link

GoogleApi.ContentWarehouse.V1.Model.AnchorsRedundantAnchorInfo

NOTE: in docjoins, the following anchor sampling information is only ## available in the first record of a document (under the same docid). The total number of redundant anchors dropped per (domain, text) in linkextractor. If we receive a large number of anchors from a particular domain, then we’ll throw out all but a sampling of them from that domain. The data is sorted by the (domain,text) pairs. This field is not populated by Alexandria, which uses cdoc.anchor_stats instead.

GoogleApi.ContentWarehouse.V1.Model.AppsDynamiteCustomerId

Represents a GSuite customer ID. Obfuscated with CustomerIdObfuscator.

GoogleApi.ContentWarehouse.V1.Model.AppsDynamiteSharedOrganizationInfo

Contains info about the entity that something is, or is owned by.

GoogleApi.ContentWarehouse.V1.Model.AppsDynamiteSharedOrganizationInfoConsumerInfo

Intentionally empty. Used to disambiguate consumer and customer use cases in oneof below.

GoogleApi.ContentWarehouse.V1.Model.AppsDynamiteSharedOrganizationInfoCustomerInfo

Attributes_12

customerId (type: GoogleApi.ContentWarehouse.V1.Model.AppsDynamiteCustomerId.t, default: nil) –

GoogleApi.ContentWarehouse.V1.Model.AppsDynamiteSharedSegmentedMembershipCount

Contains info on membership count for member types: HUMAN_USER, APP_USER & ROSTER_MEMBER different states: INVITED, JOINED

GoogleApi.ContentWarehouse.V1.Model.AppsDynamiteSharedSegmentedMembershipCounts

Attributes_13

value (type: list(GoogleApi.ContentWarehouse.V1.Model.AppsDynamiteSharedSegmentedMembershipCount.t), default: nil) –

GoogleApi.ContentWarehouse.V1.Model.AppsPeopleActivityBackendDestinationStream

A DestinationStream is a /namespace/id[0]/id[1]/…/id[n] that represents a collection of Activities. Example destinations: -The Profile Stream on http://plus.google.com/+JohnDoe/posts -A Square Stream on http://plus.google.com/squares/123 -A “comment Stream” (Fountain) on http://www.youtube.com/watch?id=123 It’s possible for a single Activity to show in each of these destinations – and it might behave/look slightly differently for each one. Destinations can have their own business logic associated with them at both write-time and read-time server-side (these are documented below). Each DestinationStream is indexed and can be retrieved using the GetDestinationStreamRequest. For the repeated ID space indexing happens at all levels, e.g. if you have: /square/123/abc /square/123/efd /square/456 You can fetch /square/123/abc directly or /square/123 (which includes all Activities in both /square/123/abc and /square/123/efd), or even /square which retrieves all Activities in the Square namespace (visible for that user). On the storage layer, we represent DestinationStream as Channel (http://cs/#google3/social/common/channel/channel.proto), since the storage does not have the concept of a Stream. Both terms are used interchangeably within the service layer, but client of Social Infrastructure should use the term DestinationStream. Next ID: 3

GoogleApi.ContentWarehouse.V1.Model.AppsPeopleActivityStreamqualityDistillerEngagements

Stores the number of different kind of user engagement actions. Abuse Report is also consider an engagement. Currently we only have abuse report engagements but in future we might add other types of engagements as well.

GoogleApi.ContentWarehouse.V1.Model.AppsPeopleOzExternalMergedpeopleapiAbout

GoogleApi.ContentWarehouse.V1.Model.AppsDynamiteSharedSegmentedMembershipCounts

Attributes_14

• contentType (type: String.t, default: nil) –
• metadata (type: GoogleApi.ContentWarehouse.V1.Model.AppsPeopleOzExternalMergedpeopleapiPersonFieldMetadata.t, default: nil) –
• safeHtmlValue (type: GoogleApi.ContentWarehouse.V1.Model.WebutilHtmlTypesSafeHtmlProto.t, default: nil) – Sanitized HTML value that is only populated when the SANITIZE_ABOUT_HTML extension is requested.
• value (type: String.t, default: nil) –

GoogleApi.ContentWarehouse.V1.Model.AppsPeopleOzExternalMergedpeopleapiAboutMeExtendedData

Extension data for use in AboutMe.

GoogleApi.ContentWarehouse.V1.Model.AppsPeopleOzExternalMergedpeopleapiAboutMeExtendedDataNameDisplayOptions

See NameDisplayOptions in //depot/google3/focus/backend/proto/backend.proto. See also go/nickname-mess.

GoogleApi.ContentWarehouse.V1.Model.AppsPeopleOzExternalMergedpeopleapiAboutMeExtendedDataPhotosCompareData